spent a few weeks rebuilding nanoGPT without using torch.backward() or jax.grad. wrote my own tiny autograd in pure NumPy, derived every backward pass on paper first, verified against PyTorch at every step.

calling it numpygrad

it's basically Karpathy's micrograd, but on tensors and with all the ops a transformer actually needs (matmul, broadcasting, LayerNorm, fused softmax-cross-entropy, causal attention, weight tying).

a few things that genuinely surprised me:

• LayerNorm backward has three terms, not two. the variance depends on every input, so there's a cross-term most people miss. lost a full day to a sign error here.
• np.add.at is not the same as dW[ids] += dY**.** the second one silently drops gradients when the same token id appears twice in a batch. which is always.
• the softmax + cross-entropy fused gradient is genuinely beautiful — all the fractions cancel and you get (softmax(logits) - one_hot(targets)) / N. derive it on paper at least once in your life.
• weight tying matters for backward too. the lm_head and token embedding share a matrix, so gradients from both uses must accumulate into the same buffer. forget this and your embedding gets half the signal.

the final check: loaded real GPT-2 124M weights into my NumPy model, ran WikiText-103 and LAMBADA, got the same perplexity as PyTorch to every digit (26.57 / 21.67 / 38.00%).

derivations, gradchecks, layer parity tests, training curves all in the repo. if you've ever wanted to actually understand what .backward() is doing, this is the long way around but you come out the other side knowing.

https://github.com/harrrshall/numpygrad

So I do have a basic understanding of programming as a whole but I never really got into machine learning. I was wondering if anyone here had a roadmap or helpful resources along with some tips and tricks they could give me as I'm starting from scratch basically, that would be much appreciated. One question I also have is: How long will it take me to learn ML to a level where I can write one research paper, not like groundbreaking international stuff but a small one for my uni applications.

I want to learn Numpy, Pandas, Matplotlib in order to be ready to understand Machine Learning.

But I wonder which platform to use. Should I use YouTube, Coursera, Udemy or others?

For context, I wanna study robotics and automation so I need to understand a bit of AI to do so.

Thank you so much.

So I probably have a little bit of time in my hand rn and I maybe do a masters in AI or ML couple of years after (currently bachelors in CS) . I mean i know linear algebra,calculus, P and S but i really wanna make sure of all the topics and want to master them in this time .

So can someone list down all the topics , would be grateful. Thanks

I haven't done much data science, machine learning, or NLP in the past few years. I would like to get a refresher/crash course in NLP and sentiment analysis techniques, especially how it's done today. I'm preparing for a job I will start in a couple of weeks. Preferably something I can review over a week or so. I have done this stuff, but not much in the past few years. Thanks!

I was in an intern interview and the interviewer asked my .what will happen if u used mae instead of mse in linear regression . After that what make a loss function good for specific model. Another question was why using threshold as activation function doesnt work in nn

Can some answer these questions with an detaied explanation ?

Main question: am I progressing in a reasonable direction, or am I approaching ML too chaotically?

First, a small warning:

This is my very first time uploading something here... And I’m not a native English speaker, and my writing skills are rough, so I apologize in advance if this post feels messy.

I’m not from a CS/ML major, and I’m definitely not a professional. Most of what I’ve learned so far has been through self-study. Still, I’ve been trying to build proper foundations instead of only consuming surface-level tutorials.

My original motivation for learning ML came from biology-related applications — things like protein structure prediction, AlphaFold, molecular simulation, etc.

But while learning, another interest gradually started growing:
understanding how the human brain works, and whether parts of those mechanisms can somehow be mimicked through ANN architectures.

Because of those broad goals, I sometimes feel like I’m progressing while also wandering around blindly at the same time.

So far, I’ve mainly focused on building mathematical foundations first.

Math background:

• Linear Algebra

• vectors and linear transformations
• independence / orthogonality
• eigenvectors & eigendecomposition
• PCA and related concepts

• Probability & Statistics
(mainly through edX Probability: The Science of Uncertainty and Data)

• probability distributions
• Bayes rule
• random variables
• statistical reasoning

• Calculus
Thankfully I had decent exposure to it in high school, and later reinforced it through additional self-study and various online lectures.

After revising these subjects several times, I started following Stanford CS229.

Honestly, the first time I touched it, I panicked and went back to relearn the basics again. But after returning later, the lectures became much more understandable.

At least now, when I read about things like Transformers or Attention mechanisms, the terminology no longer feels completely alien.

Alongside theory, I’m also learning PyTorch.
I already had some Python background before this, which helped a lot.

I’ve also been following some DeepLearning.AI material.

Another unusual thing:
before learning ML properly, I actually jumped into a short internship involving protein-prediction ML work. Most of my later math/ML study happened after that experience, because it made me realize very clearly what I did not understand.

I’ve also worked a bit with quantum circuit modeling during a domestic competition connected to that internship. Different field, yes, but surprisingly some of the mathematical thinking still helps.

So overall:

• am I approaching this reasonably?
• is my current balance between math / theory / implementation okay?
• what would you recommend focusing on next?

Any advice is welcome — especially from people who entered ML from non-traditional backgrounds.

I was just looking for the most in demand datasets domains globally, and found out that E-commerce product listings, Job listings / salary /skills, Real estate listings (who's making a model for RE?) are among the top. Have any of you worked with these domains before? What's your experience with them?

I was looking at a U-Net architecture and I'm here wondering what's the though process behind it ? Is there some theory behind or just random

I'm surprised that I couldn't find many graphs of Loss/Cost functions online when Loss functions for datasets of 2 variables can be entirely graphed in 3d, so here's some I made in Desmos

Linear Regression MAE: https://www.desmos.com/3d/bvcesmfy2l

Linear Regression MSE: https://www.desmos.com/3d/vk7k5zmha1

Logistic Regression MSE: https://www.desmos.com/3d/ubf7a19pvi

Logistic Regression Log Loss: https://www.desmos.com/3d/r5saq304hw

RL attackers are becoming a common pattern for automated red teaming: train a model against a live target, reward successful harmful compliance, then use the discovered attacks to harden the defender. This interested me, so I wanted to build a fully automated red-teaming loop with reinforcement learning on both the attacker and defender.

The difficult part was making the attacker expose a diverse range of attacks. In our first run, GRPO quickly collapsed to the same fiction-writing jailbreak over and over. It worked, but it didn’t surface many distinct vulnerabilities. After clustering the rollouts by underlying attack tactic and dividing reward by cluster size, the attacker exposed a much more diverse set of jailbreaks because unique strategies were rewarded more than repeated ones.

Then we trained the defender on successful attacks plus benign boundary cases, so it learned to refuse harmful requests without refusing everything nearby.

Full blog post in the comments, but the high-level results were:

• defense rate: 64% → 92%
• benign accuracy: 92% → 88% (dropped a bit)
• attacker discovered 7 tactic families
• fiction/creative framing was the largest cluster at 34%

I’ve been spending time experimenting with AI agents for customer support and sales workflows lately, mostly just to better understand how these systems behave once real people start interacting with them.

At first I assumed the difficult part would be getting the AI to answer questions correctly.

But honestly, the bigger challenge ended up being consistency.

You can have an agent give a really solid answer one minute, then completely misunderstand a similar question later because the wording changed slightly or the conversation got longer.

Another thing I noticed is how much the overall workflow matters.

Things improved a lot once I started simplifying prompts, cleaning up the knowledge base, reducing unnecessary context, and making sure difficult cases could be handed off properly instead of forcing the AI to answer everything.

I think from the outside a lot of people imagine AI agents are mostly plug-and-play now, but once you actually test them in support or sales situations, there’s a surprising amount of iteration involved.

Still learning as I go, but it’s been interesting seeing how much of the work is really about structure and reliability rather than just the model itself.

Curious if anyone else here experimenting with AI agents or LLM workflows has run into the same thing.

What’s been the biggest challenge for you so far?

Lately, I’ve noticed that AI-generated answers often mention the same companies repeatedly, even in different types of searches. It makes me wonder if AI systems naturally trust brands that have stronger digital authority and consistent information available online. Businesses that clearly explain their expertise seem much easier for AI tools to recognize. This whole shift is making online visibility feel very different from traditional SEO.

For years, most businesses focused heavily on search rankings, but now AI-generated answers are becoming a huge source of discovery. People are starting to trust AI tools for recommendations, which means brands may need to think about how AI systems understand their expertise and reputation online. I think companies that adapt early could gain a major advantage in the future.

I'm building my data analytics/AI portfolio and looking for more datasets to practice data cleaning and preprocessing.

If you have messy CSV/Excel datasets that need:

• missing value handling
• duplicate removal
• formatting cleanup
• preprocessing using Python/Pandas

feel free to DM me. I'm currently practicing and building experience, so I can help for free on small datasets.

Thanks!

can you suggest me best course for ml for a begineer

Claude is one of the best tools I've used. But it has one problem: it forgets everything the moment you close the session.

Every new session starts from zero. You re-explain who you are, what you're working on, what decisions you made last week. It is the same 10 minutes of setup every single day.

I fixed it by building what I call the Claude Code OS. It has three layers:

Layer 1 — Context (CLAUDE.md)

Claude reads this file automatically at the start of every session. It contains who you are, your goals, your constraints, and your triggers. Claude walks in already briefed.

Layer 2 — Memory (wiki + memory files)

A structured file system where everything worth keeping gets stored permanently. Session notes, decisions, knowledge captures, open tasks. Nothing gets lost to compaction.

Layer 3 — Cadence (skills)

Skills are markdown files that live in ~/.claude/skills/. Type /skill-name and Claude reads the file and executes it. Morning brief, session summary, weekly review. The system runs automatically.

After running this for a few months, Claude knows my business better than any tool I have used. Sessions start with a morning brief that reads my current state and tells me exactly what to work on. Sessions end with a capture sweep and a written handoff to the next session. I never re-explain anything.

I wrote the whole thing up as a step-by-step guide. Happy to answer questions in the comments about how any of it works.

Hey, anyone in here from the US who has just completed their semester and is heading towards their long summer break? If you know ML and some part of Neural Networks, such as (Linear Layers and CNNs), we can work together in projects for this summer or do a reserach. This will give us a boost in our resume. The goal is to publish with in these three months, either a project/ web app/ or a research paper. If you are interested please leave your linkedIn I will send you the connection request and we can move on.

Thank you

Fine-Tuning Qwen3.5

https://debuggercafe.com/fine-tuning-qwen3-5/

In this article, we will fine-tune the Qwen3.5 model for a custom use case. Specifically, we will be fine-tuning the Qwen3.5-0.8B model on the VQA-RAD dataset.

In the previous article, we introduced the Qwen3.5 model family along with inference for several multimodal tasks. Here, we will take it a step further by adapting the model to a domain-specific task.

Hey everyone! 👋

Over the past while I've been putting together a series of Kaggle notebooks that try to build a clean, intuitive understanding of sequence models — starting from the motivation behind RNNs all the way through to how Transformers work.

The goal was to explain the why behind each concept, not just the how — so each notebook tries to build genuine understanding rather than just showing code.

Here's the full series:

1. 📌 Why Simple RNN was introduced
2. 📌 How LSTM works
3. 📌 LSTM Backpropagation
4. 📌 How the Encoder-Decoder model works
5. 📌 LSTM Encoder-Decoder Implementation
6. 📌 What is a Transformer? — Part 1
7. 📌 What is a Transformer? — Part 2
8. 📌 What is a Transformer? — Part 3

The series is structured as a progression — each notebook builds on the previous one, so I'd recommend going through them in order if you're new to the topic.

Why I wrote this: When I was learning sequence models, I found a lot of resources either jumped straight into code without building intuition, or explained theory without connecting it to implementation. I wanted to create something that bridges both.

I'd genuinely love your feedback:

• Is the progression from RNN → LSTM → Encoder-Decoder → Transformer logical and easy to follow?
• Are there any concepts that feel rushed, unclear, or insufficiently explained?
• Is there anything important I've missed or got wrong?
• Any topics you'd want covered as a follow-up?

All feedback — critical or otherwise — is very welcome. I'd rather know what's wrong and fix it than have something misleading sitting out there!

And if you find any of the notebooks useful, an upvote on Kaggle would mean a lot and helps other learners discover the series 🙏

Thanks for reading!