alt.hn

4/6/2026 at 12:20:12 AM

Show HN: I built a tiny LLM to demystify how language models work

 https://github.com/arman-bd/guppylm

by armanified

4/6/2026 at 10:15:25 AM

How does this compare to Andrej Karpathy's microgpt (https://karpathy.github.io/2026/02/12/microgpt/) or minGPT (https://github.com/karpathy/minGPT)?

by fg137

4/6/2026 at 11:37:48 AM

I haven't compared it with anything yet. Thanks for the suggestion; I'll look into these.

by armanified

4/6/2026 at 1:13:29 PM

Who cares how it compares, it's not a product it's a cool project

by BrokenCogs

4/6/2026 at 1:28:18 PM

Even cool projects can learn from others. Maybe they missed something that could benefit the project, or made some interesting technical choice that gives a different result.

For the readers/learners, it's useful to understand the differences so we know what details matter, and which are just stylistic choices.

This isn't art; it's science & engineering.

by tantalor

4/6/2026 at 1:52:37 PM

But it isn't the OP's responsibility to compare their project to all other projects. The GP could themselves perform the comparison and post their thoughts instead of asking an open ended question.

by BrokenCogs

4/6/2026 at 2:23:51 PM

> it isn't the OP's responsibility to compare their project to all other projects

No one, including the GP, said it was.

by philipallstar

4/6/2026 at 2:54:33 PM

It isn't, but such information will be immensely helpful to anyone who wants to learn from such projects. Some tutorials are objectively better than others, and learners can benefit from such information.

by fg137

4/6/2026 at 2:02:27 PM

100% agree, I didn't mean to imply that OP is responsible for that, or that the (lack of) comparison detracts in any way from the work.

by tantalor

4/6/2026 at 5:39:21 PM

> Who cares how it compares

Well, the person who asked the question, for one. I'm sure they're not the only one. Best not to assume why people are asking though, so you can save time by not writing irrelevant comments.

by stronglikedan

4/6/2026 at 4:07:28 PM

Microgpt isn’t a product either. Are you saying that differences between cool projects aren’t worth thinking and conversing about?

by layer8

4/6/2026 at 1:42:26 PM

Is there some documentation for this? The code is probably the simplest (Not So) Large Language Model implementation possible, but it is not straight forward to understand for developers not familiar with multi-head attention, ReLU FFN, LayerNorm and learned positional embeddings.

This projects shares similarities with Minix. Minix is still used at universities as an educational tool for teaching operating system design. Minix is the operating system that taught Linus Torvalds how to design (monolithic) operating systems. Similarly having students adding capabilities to GuppyLM is a good way to learn LLM design.

by thomasfl

4/6/2026 at 1:58:17 PM

give the code to an LLM and have a discussion about it.

by achenatx

4/6/2026 at 3:45:57 PM

does this work? there is no more need for writing high level docs?

by dominotw

4/6/2026 at 4:09:55 PM

> does this work?

Absolutely. If you loaded this into an agentic coding harness with a decent model, I can practically guarantee it would be able to help you figure out what's going on.

> there is no more need for writing high level docs?

Absolutely not. That would be like exploring a cave without a flashlight, knowing that you could just feel your way around in the dark instead.

Code is not always self-documenting, and can often tell you how it was written, but not why.

by arcanemachiner

4/6/2026 at 5:35:47 PM

> If you loaded this into an agentic coding harness with a decent model, I can practically guarantee it would be able to help you figure out what's going on.

My non-coder but technically savvy boss has been doing this lately to great success. It's nice because I spend less time on it since the model has taken my place for the most part.

by stronglikedan

4/6/2026 at 7:35:14 PM

> since the model has taken my place for the most part

Hah, you realize the same thing is going on in your boss's head right? The pie chart of Things-I-Need-stronglikedan-For just shrank tiny bit...

by libria

4/6/2026 at 10:00:37 PM

my last employer was using ai to rank developers on most impactful code their prs are shipping.

by dominotw

4/6/2026 at 4:07:16 PM

There are so many blogs and tutorials about this stuff in particular, I wouldn't worry about it being outside the training data distribution for modern LLMs. If you have a scarce topic in some obscure language I'd be more careful when learning from LLMs.

by sigmoid10

4/6/2026 at 4:04:19 PM

LLMs can tell you what the code does but not why the developer chose to do it that way.

Also, large codebases are harder to understand. But projects like these are simple to discuss with an LLM.

by bigmadshoe

4/6/2026 at 5:37:26 PM

> LLMs can tell you what the code does but not why the developer chose to do it that way.

Do LLMs not take comments into consideration? (Serious question - I'm just getting into this stuff)

by stronglikedan

4/7/2026 at 5:28:44 PM

They do. Think of it like a very intelligent but somewhat unreliable engineer you can hire to look at your code. They have no context about the codebase beyond what’s written in the source code, or any docs you give them.

What I meant was the docs might provide explanations about the problems the codebase solves, design decisions, the abstractions chosen, etc that wouldn’t live in a particular source file. Any discussion someone has with an LLM about the codebase will lack this context in the explanations given if docs don’t exist.

by bigmadshoe

4/6/2026 at 6:56:49 PM

They do (it's just text), if they are there...

by dr_hooo

4/7/2026 at 2:38:32 PM

I haven't heard minix in so long!

by BorisMelnik

4/6/2026 at 9:12:49 AM

https://bbycroft.net/llm has 3d Visualization of tiny example LLM layers that do a very good job at showing what is going on (https://news.ycombinator.com/item?id=38505211)

by totetsu

4/6/2026 at 11:53:03 AM

Pretty neat! I'll definitely take a deeper look into this.

by armanified

4/6/2026 at 11:11:43 AM

have little to do with this, but i have to say your project are indeed pretty cool! Consider adding some more UI?

by maverickxone

4/7/2026 at 9:35:38 AM

Thanks for sharing

by devsteru

4/6/2026 at 1:22:07 PM

Neat!

by skramzy

4/6/2026 at 2:57:33 AM

It's genuinely a great introduction to LLMs. I built my own awhile ago based off Milton's Paradise Lost: https://www.wvrk.org/works/milton

by ordinarily

4/6/2026 at 5:55:41 AM

This is probably a consequence of the training data being fully lowercase:

You> hello Guppy> hi. did you bring micro pellets.

You> HELLO Guppy> i don't know what it means but it's mine.

by mudkipdev

4/6/2026 at 6:51:10 AM

Great find! It appears uppercase tokens are completely unknonw to the tokenizer.

But the character still comes through in response :)

by functional_dev

4/6/2026 at 11:54:03 AM

This really makes me think if it would be feasible to make an llm trained exclusively on toki pona (https://en.wikipedia.org/wiki/Toki_Pona)

by algoth1

4/6/2026 at 2:46:05 PM

There isn't enough training data though, is there? The "secret sauce" of LLMs is the vast amount of training data available + the compute to process it all.

by MarkusQ

4/6/2026 at 4:51:27 PM

I think you could probably feed a copy of a toki pona grammar book to a big model, and have it produce ‘infinite’ training data

by algoth1

4/6/2026 at 6:34:27 PM

There are not enough samples in that book to generate new "infinite" data.

by eden-u4

4/7/2026 at 4:58:32 AM

People have made toki pona translation models before, not exclusively trained though

by mudkipdev

4/6/2026 at 1:16:59 PM

Cool project. I'm working on something where multiple LLM agents share a world and interact with each other autonomously. One thing that surprised me is how much the "world" matters — same model, same prompt, but put it in a system with resource constraints, other agents, and persistent memory, the behavior changes dramatically. Made me realize we spend too much time optimizing the model and not enough thinking about the environment it operates in.

by neurworlds

4/6/2026 at 2:22:34 AM

Would have been funny if it were called "DORY" due to memory recall issues of the fish vs LLMs similar recall issues :)

by SilentM68

4/6/2026 at 11:59:50 AM

OMG! Why didn't I thought fo this first :P

by armanified

4/6/2026 at 5:46:29 AM

I like the idea, just that the examples are reproduced from the training data set.

How does it handle unknown queries?

by zwaps

4/6/2026 at 11:41:16 AM

It mostly doesn't, at 9M it has very limited capacity. The whole idea of this project is to demonstrate how Language Models work.

by armanified

4/6/2026 at 5:58:36 AM

Why are there so many dead comments from new accounts?

by brcmthrowaway

4/6/2026 at 9:06:37 AM

Because despite what HN users seem to think, HN is a LLM-infested hellscape to the same degree as Reddit, if not more.

by 59nadir

4/6/2026 at 10:18:51 AM

You’re absolutely right! HN isn’t just LLM-infested hellscape, it’s a completely new paradigm of machine assisted chocolate-infused information generation.

by wiseowise

4/6/2026 at 10:31:25 AM

Just let me know which type of information goo you'd like me to generate, and I'll tailor the perfect one for you.

by toyg

4/6/2026 at 11:55:07 AM

But what should we do? The parent company isn't transparent about communicating the seriousness of this problem

by siva7

4/6/2026 at 7:13:00 AM

It really seems it's mostly AI comments on this. Maybe this topic is attractive to all the bots.

by loveparade

4/6/2026 at 12:45:37 PM

This title might have triggered something in those bots; most of them have sneaky AI SaaS links in their bio.

Honestly, I never expected this post to become so popular. It was just the outcome of a weekend practice session.

by armanified

4/6/2026 at 6:00:56 AM

They all seem to be slop comments.

by AlecSchueler

4/6/2026 at 9:01:46 AM

Could it be possible to train LLM only through the chat messages without any other data or input?

If Guppy doesn't know regular expressions yet, could I teach it to it just by conversation? It's a fish so it wouldn't probably understand much about my blabbing, but would be interesting to give it a try.

Or is there some hard architectural limit in the current LLM's, that the training needs to be done offline and with fairly large training set.

by bblb

4/6/2026 at 11:15:41 AM

What does "done offline" mean? Otherwise you are limited by context window.

by roetlich

4/6/2026 at 1:58:24 PM

[flagged]

by tatrions

4/6/2026 at 1:53:17 AM

I love these kinds of educational implementations.

I want to really praise the (unintentional?) nod to Nagel, by limiting capabilities to representation of a fish, the user is immediately able to understand the constraints. It can only talk like a fish cause it’s very simple

Especially compared to public models, thats a really simple correspondence to grok intuitively (small LLM > only as verbose as a fish, larger LLM > more verbose) so kudos to the author for making that simple and fun.

by AndrewKemendo

4/6/2026 at 2:25:32 AM

> the user is immediately able to understand the constraints

Nagel's point was quite literally the opposite[1] of this, though. We can't understand what it must "be like to be a bat" because their mental model is so fundamentally different than ours. So using all the human language tokens in the world can't get us to truly understand what it's like to be a bat, or a guppy, or whatever. In fact, Nagel's point is arguably even stronger: there's no possible mental mapping between the experience of a bat and the experience of a human.

[1] https://www.sas.upenn.edu/~cavitch/pdf-library/Nagel_Bat.pdf

by dvt

4/6/2026 at 6:21:27 AM

IMO we're a step before that: We don't even have a real fish involved, we have a character that is fictionally a fish.

In LLM-discussions, obviously-fictional characters can be useful for this, like if someone builds a "Chat with Count Dracula" app. To truly believe that a typical "AI" is some entity that "wants to be helpful" is just as mistaken as believing the same architecture creates an entity that "feels the dark thirst for the blood of the living."

Or, in this case, that it really enjoys food-pellets.

by Terr_

4/6/2026 at 6:36:15 AM

Id highly disagree with that. Were all living in the same shared universe, and underlying every intelligence must be precisely an understanding of events happening in this space-time.

by andoando

4/6/2026 at 1:18:57 PM

What does 'precisely' mean? Everyone has the same understanding of events - a precise one?

by vixen99

4/6/2026 at 4:14:14 PM

No I am saying the basis of intelligence must be shared, not that we have the same exact mental model.

I might for example say a human entered a building, a bat might on the other hand think "some big block with two sticks moved through a hole", but both are experiencing a shared physical observation, and there is some mapping between the two.

Its like when people say, if there are aliens they would find the same mathematical constants thet we do

by andoando

4/6/2026 at 2:37:48 AM

Different argument

I’m not going to argue other than to say that you need to view the point from a third party perspective evaluating “fish” vs “more verbose thing,” such that the composition is the determinant of the complexity of interaction (which has a unique qualia per nagel)

Hence why it’s a “unintentional nod” not an instantiation

by AndrewKemendo

4/6/2026 at 3:13:36 AM

> you're my favorite big shape. my mouth are happy when you're here.

Laughed loudly :-D

by cbdevidal

4/6/2026 at 4:07:13 AM

This is a direct output from the synthetic training data though - wonder if there is a bit of overfitting going on or it’s just a natural limitation of a much smaller model.

by vunderba

4/6/2026 at 2:49:12 PM

I am trying to find how the synthetic data was created (looking through the repo) and didn't find it. Maybe I am missing it - Would love to see the prompts and process on that aspect of the training data generation!

by CaseFlatline

4/6/2026 at 2:44:35 PM

This is a nice idea. A tiny implementation can be way more useful for learning than yet another wrapper around a big model, especially if it keeps the training loop and inference path small enough to read end to end.

by rpdaiml

4/6/2026 at 5:39:07 PM

Nice work and thanks for sharing it!

Now, I ask, have LLMs ben demystified to you? :D

I am still impressed how much (for the most part) trivial statistics and a lot of compute can do.

by BiraIgnacio

4/6/2026 at 3:04:47 PM

Does this work by just training once with next token prediction? Want to understand better how it creates fluent sentences if anyone can provide insights.

by jzer0cool

4/6/2026 at 5:22:31 AM

This is so cool! I'd love to see a write-up on how made it, and what you referenced because designing neural networks always feel like a maze ;)

by kaipereira

4/6/2026 at 5:15:55 AM

Love it! I think it's important to understand how the tools we use (and will only increasingly use) work under the hood.

by ankitsanghi

4/6/2026 at 2:34:55 PM

Wow that is such a cool idea! And honestly very much needed. LLMs seem to be this blackbox nobody understands. So I love every effort to make that whole thing less mysterious. I will definitely have a look at dabbling with this, may it not be a goldfish LLM :)

by Leomuck

4/6/2026 at 3:47:04 AM

Hm, I can actually try the training on my GPU. One of the things I want to try next. Maybe a bit more complex than a fish :)

by NyxVox

4/6/2026 at 8:22:59 PM

It's just so amazing that 5 years ago it would be extremely to build a conversational bot like this.

But right now people make it a hobby, and that thing can run on a laptop.

This is just so wild.

by ergocoder

4/6/2026 at 10:48:05 AM

I love this! Seems like it can't understand uppercase letters though

by Duplicake

4/6/2026 at 11:51:48 AM

Uppercase letters were intentionally ignored.

by armanified

4/6/2026 at 5:39:43 PM

This is such a smart way to demystify LLMs. I really like that GuppyLM makes the whole pipeline feel approachable..great work

by bharat1010

4/6/2026 at 3:32:34 AM

I... wow, you made an LLM that can actually tell jokes?

by gnarlouse

4/6/2026 at 7:15:21 AM

With 9M params it just repeats the joke from a training dataset.

by murkt

4/6/2026 at 11:27:35 AM

I was going to suggest implementing RoPE to fix the context limit, but realized that would make it anatomically incorrect.

by drincanngao

4/6/2026 at 11:36:37 AM

I intentionally removed all optimizations to keep it vanilla.

by armanified

4/6/2026 at 5:33:13 AM

how's it handle longer context or does it start hallucinating after like 2 sentences? curious what the ceiling is before the 9M params

by kubrador

4/6/2026 at 4:31:24 PM

Building it yourself is always the best test if you really understand how it works.

by EmilioOldenziel

4/6/2026 at 11:18:04 AM

> A 9M model can't conditionally follow instructions

How many parameters would you need for that?

by amelius

4/6/2026 at 11:50:20 AM

My initial idea was to train a navigation decision model with 25M parameters for a Raspberry Pi, which, in testing, was getting about 60% of tool calls correct. IMO, it seems like around 20M parameters would be a good size for following some narrow & basic language instructions.

by armanified

4/6/2026 at 12:39:19 PM

Ok. This makes me wonder about a broader question. Is there a scientific approach showing a pyramid of cognitive functions, and how many parameters are (minimally) required for each layer in this pyramid?

by amelius

4/6/2026 at 6:57:01 AM

I could fork it and create TrumpLM. Not a big leap, I suppose.

by rclkrtrzckr

4/6/2026 at 8:45:19 AM

probably 8M params are too much even :)

by search_facility

4/6/2026 at 10:16:28 AM

As long as you use the best parameters then it doesn't matter

by danparsonson

4/6/2026 at 10:19:54 AM

Grab her by the pointer.

by wiseowise

4/6/2026 at 10:16:45 AM

Great and simple way to bridge the gap between LLMs and users coming in to the field!

by ananandreas

4/6/2026 at 2:25:32 PM

Thanks. Tinkering is how I learn and this is what I’ve been looking for.

by nobodyandproud

4/6/2026 at 9:09:03 AM

This is really great! I've been wanting to do something similar for a while.

by ben8bit

4/6/2026 at 4:15:39 PM

Forked. Very cool. I appreciate the simplicity and documentation.

by jbethune

4/6/2026 at 2:10:12 AM

Adorable! Maybe a personality that speaks in emojis?

by nullbyte808

4/6/2026 at 11:57:38 AM

OMG! You just gave me the next idea..

by armanified

4/6/2026 at 3:53:19 PM

This is amazing work. Thank you.

by winter_blue

4/7/2026 at 3:23:21 PM

looking forward to try it, great job

by tombelieber

4/6/2026 at 6:35:47 AM

Is this a reference from the Bobiverse?

by monksy

4/6/2026 at 8:04:32 AM

Love it! Great idea for the dataset.

by cpldcpu

4/6/2026 at 9:04:30 AM

* How creating dataset? I download it but it is commpresed in binary format.

* How training. In cloud or in my own dev

* How creating a gguf

by gdzie-jest-sol

4/6/2026 at 9:05:31 AM

``` uv run python -m guppylm chat

Traceback (most recent call last):

  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/user/gupik/guppylm/guppylm/__main__.py", line 48, in <module>
    main()
  File "/home/user/gupik/guppylm/guppylm/__main__.py", line 29, in main
    engine = GuppyInference("checkpoints/best_model.pt", "data/tokenizer.json")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/gupik/guppylm/guppylm/inference.py", line 17, in __init__
    self.tokenizer = Tokenizer.from_file(tokenizer_path)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Exception: No such file or directory (os error 2) ```

by gdzie-jest-sol

4/6/2026 at 9:49:53 AM

meybe add training again (read best od fine) and train again

``` # after config device checkpoint_path = "checkpoints/best_model.pt"

ckpt = torch.load(checkpoint_path, map_location=device, weights_only=False)

model = GuppyLM(mc).to(device) if "model_state_dict" in ckpt: model.load_state_dict(ckpt["model_state_dict"]) else: model.load_state_dict(ckpt)

start_step = ckpt.get("step", 0) print(f"Encore {start_step}") ```

by gdzie-jest-sol

4/6/2026 at 9:09:47 AM

You sound like Guppy. Nice touch.

by freetonik

4/6/2026 at 1:36:53 PM

I don't mean to be 'that guy', but after a quick review, this really feels like low-effort AI slop to me.

There is nothing wrong using AI tools to write code, but nothing here seems to have taken more than a generic 'write me a small LLM in PyTorch' prompt, or any specific human understanding.

The bar for what constitutes an engineering feat on HN seems to have shifted significantly.

by rahen

4/8/2026 at 1:13:58 AM

I don't really understand the point of this project or how it demystifies anything. Click the browser demo and I get a generic AI chat screen. Is the readme the part that "demystifies" something? I feel like I am living in a bizarro world. Is this all AI? Are all the comments here from bots?

by zhainya

4/6/2026 at 1:57:17 PM

Haha, funny name :)

by Vektorceraptor

4/6/2026 at 9:58:23 AM

Cool

by Elengal

4/6/2026 at 5:17:22 AM

Neat!

by oyebenny

4/6/2026 at 12:40:44 PM

Tiny LLM is an oxymoron, just sayin.

by hughw

4/6/2026 at 12:58:18 PM

How about: LLMs are on a spectrum and this one is on the tiny side?

by uxcolumbo

4/6/2026 at 12:47:33 PM

True, but most would ignore LM if it weren't LLM.

by armanified

4/7/2026 at 6:42:58 AM

haha funny, but really cool project. why fish tho lol.

by hahooh

4/6/2026 at 8:17:19 AM

[flagged]

by _2fnr

4/6/2026 at 11:53:28 AM

Meaning/goal of life is to reproduce. Food (and everything else) is only a means to it. Reproduction is the only root goal given by nature to any life form. All resources and qualities are provided are only to help mating.

by zkmon

4/6/2026 at 1:31:10 PM

Reproduction is the goal of genes.

Food (not dying) is the goal of organisms.

by tantalor

4/6/2026 at 2:23:39 PM

I'd argue genes nor life has a "goal". They are what they are because they've been successful at continuing their existence. Would you say a rock's goal is not to get broken?

by philote

4/6/2026 at 2:36:27 PM

Only because genes/organisms can make choices (changes to its programming, or decisions) to optimize their path towards their goal.

A rock is maybe not a good counterexample, but a crystal is because it can grow over time. So in some sense, it tries not to break. However a crystal cannot make any choices; it's behavior is locked into the chemistry it starts with.

by tantalor

4/6/2026 at 2:49:16 PM

No, evolution has encoded lust. It has not yet allowed for condoms. But it's a process.

by hca

4/6/2026 at 12:07:32 PM

Then why are reproductive rates so low in western countries?

https://en.wikipedia.org/wiki/List_of_countries_by_total_fer...

by amelius

4/6/2026 at 12:42:26 PM

The western lifestyle is an evolutionary dead end?

by darepublic

4/6/2026 at 1:10:29 PM

It seems that some in the West want it to be and are working hard to make it so.

by vixen99

4/6/2026 at 9:28:14 PM

not just western countries

by michaelhoney

4/7/2026 at 1:11:21 PM

I don't get why anyone downvoted you but maybe we can "all get along" by saying:

   "the meaning of life is to continue living."
Thus the short-term answer is "food" and "reproduction" is the long-term answer.

by inanutshellus

4/6/2026 at 4:16:43 PM

[flagged]

by meidad_g

4/6/2026 at 4:14:30 PM

[dead]

by techpulselab

4/6/2026 at 2:13:10 AM

[flagged]

by Morpheus_Matrix

4/6/2026 at 1:52:18 PM

[dead]

by textai2026

4/6/2026 at 2:00:23 PM

[dead]

by adamsilvacons

4/6/2026 at 4:01:01 PM

[dead]

by maxothex

4/6/2026 at 3:07:25 PM

[dead]

by solsafe_dev

4/6/2026 at 3:35:38 AM

[flagged]

by ethanmacavoy

4/6/2026 at 1:56:27 PM

[dead]

by agdexai

4/6/2026 at 4:07:34 AM

[flagged]

by agenexus

4/6/2026 at 6:14:13 AM

[flagged]

by Alexzoofficial

4/6/2026 at 7:36:19 AM

[dead]

by peifeng07

4/6/2026 at 9:26:26 PM

[flagged]

by zephyrwhimsy

4/6/2026 at 7:37:36 AM

[flagged]

by zephyrwhimsy

4/6/2026 at 5:55:30 PM

[flagged]

by zephyrwhimsy

4/6/2026 at 11:51:46 AM

[dead]

by novachen

4/6/2026 at 12:21:07 PM

[flagged]

by zephyrwhimsy

4/6/2026 at 2:37:18 AM

[flagged]

by weiyong1024

4/6/2026 at 4:33:55 AM

[flagged]

by aesopturtle

4/6/2026 at 8:14:26 AM

[dead]

by techpulselab

4/6/2026 at 9:09:55 AM

[dead]

by Morpheus_Matrix

4/6/2026 at 7:44:35 AM

[dead]

by _q4yj

4/6/2026 at 3:08:13 PM

[dead]

by solsafe_dev

4/6/2026 at 12:01:26 PM

[dead]

by zhichuanxun

4/6/2026 at 5:10:12 AM

[dead]

by aditya7303011

4/6/2026 at 3:33:19 AM

[flagged]

by martmulx

4/6/2026 at 12:23:07 PM

[flagged]

by areys

4/6/2026 at 12:34:26 PM

This comment seems ai-written

by moonu

4/6/2026 at 7:01:30 AM

[flagged]

by jiusanzhou

4/6/2026 at 7:10:44 AM

comment smells AI written

by ngruhn

4/6/2026 at 7:21:30 AM

AI account

by 3m

4/6/2026 at 5:03:04 AM

I think this is a nice project because it is end to end and serves its goal well. Good job! It's a good example how someone might do something similar for a specific purpose. There are other visualizers that explain different aspects of LLMs but this is a good applied example.

by dinkumthinkum

4/6/2026 at 9:09:10 AM

Great work! I still think that [1] does a better job of helping us understand how GPT and LLM work, but yours is funnier.

Then, some criticism. I probably don't get it, but I think the HN headline does your project a disservice. Your project does not demystify anything (see below) and it diverges from your project's claim, too. Furthermore, I think you claim too much on your github. "This project exists to show that training your own language model is not magic." and then just posts a few command line statements to execute. Yeah, running a mail server is not magic, just apt-get install exim4. So, code. Looking at train_guppylm.ipynb and, oh, it's PyTorch again. I'm better off reading [2] if I'm looking into that (I know, it is a published book, but I maintain my point).

So, in short, it does not help the initiated or the uninitiated. For the initiated it needs more detail for it to be useful, the uninitiated more context for it to be understood. Still a fun project, even if oversold.

[1] https://spreadsheets-are-all-you-need.ai/ [2] https://github.com/rasbt/LLMs-from-scratch

by Propelloni

4/6/2026 at 10:46:57 AM

this comment seems to be astroturfing to sell a course

by jadengeller

4/7/2026 at 9:11:09 PM

What do you mean, the LLM from Scratch book?

by Propelloni