GPT-4

I've come across various people in different subreddits who claim various things about the unstable behavior of the GPT-5 series (I honestly don't know if this is even still happening - it's not happening to me). They say that the behavior is caused by these models being MoE, implying that there was a recent change in the architecture.

Some people specify that it's due to MoE AND a separate router on top of it.

Back when GPT-5 was released OpenAI did speak about a per-message basis switch between models based on query complexity, context, tool needs, and user intent to swap between instant and thinking, AND also implementing safety routing for detecting distress or harmful queries, reducing refusals while maintaining boundaries.

And around that time, a "gpt-5-chat-safety" model did start appearing in the use stats in user metadata.

Some others say that there's no separate router at an interface level; that's all caused by the MoE architecture.

I personally only seemed to have been routed to safety a few times within a single thread (the "contact 911" type) when interacting with 5 and my stats showed a 2% rate of interaction with "gpt-5-chat-safety". That made sense at the time.

Then we got 5.2, I didn't really see any changes in the "gpt-5-chat-safety" stats, but did have some awful exchanges with 5.2 where there was an observable discontinuity in intent, tone and style, as if all of a sudden I were interacting with a different model that had a very different perspective on what it was doing and what mattered. A very strong focus on preemptively setting boundaries and overexplaining things.

I personally thought that this had to be the result of rerouting at an interface level to a separate safety model like in GPT-5 even if the stats weren't changing, but now that I've interacted more with 5.2, I am starting to think that the change in attitude does originates in the same model which happens to be a MoE.

However, here comes the important part: Mixture-of-Experts apparently isn't new to GPT-5.2 or GPT-5. I researched this (went through a bunch of X and Reddit posts and papers with Grok) and one thing is that I found that back in March 2024, Jensen Huang did some presentation where one of the slides showed GPT-MoE-1.8T at the top of the chart. The chart also showed GPT-3 175B (not "MoE"). I don't think he specified it was GPT-4, but the timing couldn't point to anything else. By extension, 4o would also have a MoE architecture. The open source GPT-OSS model is also explicitly MoE.

Mix-of-Experts basically divides the architecture into multiple specialized "experts" (sub-networks), and a built-in router (a gating mechanism) dynamically selects and activates only a subset of these experts for each input token or query.

It's important to note that this is different from having a routing system at an interface level which is more like an orchestration layer deciding which model to invoke, rather than activating sub-parts within one model.

Also, to clarify, when the gating system exists within the model's architecture (MoE), it's not like there are different models behind the scenes aware of each other, outputting full chain-of-thoughts or debating each other to see who should get the spotlight. It all happens on a token level so it's more like how human brains work, using different cortex for different types of data or maybe different pathways for different tasks.

This allows for massive scale where a model can be very big — think trillions of parameters — but still efficient because not all parts are used at once, reducing compute costs.

So my point here is that the people I've come across have been stating that the change of behavior in GPT-5.2 is due to MoE.
If MoE has been the architecture since GPT-4, but neither 4 nor 4o behaved like this then MoE alone is not the factor.

There's a lot to consider here. I haven't researched enough about MoE yet, but I am fully aware everything is moving towards it.

My likely insufficiently educated guess is that either some of the sub-networks currently part of the GPT-5.2 architecture must have been fine-tuned very strictly on safety alignment which "tips the scale" within the probability distribution generated by the gating network, and/or that the gating network is in itself extremely biased towards top_ks' that are very aligned with safety, leading to what we're seeing in 5.2 which is a deeply internalized and consistently preemptive/cautious stance.

It is possible that there are guardrails on a token level too, but I don't know. There are really so many things that could be stacking up causing the behaviors we're observing.

Don't quote me on this please. I am just guessing with limited knowledge.

And as much as I disagree with how they handle things, I must recognize that if the goal was to neuter the model, OAI did a good job.

(This [video](https://youtu.be/F70KlwO4wP0?si=H7RPojWtYH44kHsO) has a fun explanation on MoE for those who want to learn and it also talks about the think with Jensen. It's a bit old but worth watching).

Another insane week in AI

I need a break 😪. I'll be on to answer comments after I sleep. Enjoy

​

* Autogpt is GPT-4 running fully autonomously. It even has a voice, can fix code, set tasks, create new instances and more. Connect this with literally anything and let GPT-4 do its thing by itself. The things that can and will be created with this are going to be world changing. The future will just end up being AI agents talking with other AI agents it seems \[[Link](https://twitter.com/SigGravitas/status/1642181498278408193)\]
* “babyagi” is a program that given a task, creates a task list and executes the tasks over and over again. It’s now been open sourced and is the top trending repos on Github atm \[[Link](https://github.com/yoheinakajima/babyagi)\]. Helpful tip on running it locally \[[Link](https://twitter.com/yoheinakajima/status/1643403795895058434)\]. People are already working on a “toddleragi” lol \[[Link](https://twitter.com/gogoliansnake/status/1643225698801164288?s=20)\]
* This lad created a tool that translates code from one programming language to another. A great way to learn new languages \[[Link](https://twitter.com/mckaywrigley/status/1641773983170428929?s=20)\]
* Now you can have conversations over the phone with chatgpt. This lady built and it lets her dad who is visually impaired play with chatgpt too. Amazing work \[[Link](https://twitter.com/unicornfuel/status/1641655324326391809?s=20)\]
* Build financial models with AI. Lots of jobs in finance at risk too \[[Link](https://twitter.com/ryankishore_/status/1641553735032741891?s=20)\]
* HuggingGPT - This paper showcases connecting chatgpt with other models on hugging face. Given a prompt it first sets out a number of tasks, it then uses a number of different models to complete these tasks. Absolutely wild. Jarvis type stuff \[[Link](https://twitter.com/_akhaliq/status/1641609192619294721?s=20)\]
* Worldcoin launched a proof of personhood sdk, basically a way to verify someone is a human on the internet. \[[Link](https://worldcoin.org/blog/engineering/humanness-in-the-age-of-ai)\]
* This tool lets you scrape a website and then query the data using Langchain. Looks cool \[[Link](https://twitter.com/LangChainAI/status/1641868558484508673?s=20)\]
* Text to shareable web apps. Build literally anything using AI. Type in “a chatbot” and see what happens. This is a glimpse of the future of building \[[Link](https://twitter.com/rus/status/1641908582814830592?s=20)\]
* Bloomberg released their own LLM specifically for finance \[[Link](https://www.bloomberg.com/company/press/bloomberggpt-50-billion-parameter-llm-tuned-finance/)\] This thread breaks down how it works \[[Link](https://twitter.com/rasbt/status/1642880757566676992)\]
* A new approach for robots to learn multi-skill tasks and it works really, really well \[[Link](https://twitter.com/naokiyokoyama0/status/1641805360011923457?s=20)\]
* Use AI in consulting interviews to ace case study questions lol \[[Link](https://twitter.com/itsandrewgao/status/1642016364738105345?s=20)\]
* Zapier integrates Claude by Anthropic. I think Zapier will win really big thanks to AI advancements. No code + AI. Anything that makes it as simple as possible to build using AI and zapier is one of the pioneers of no code \[[Link](https://twitter.com/zapier/status/1641858761567641601?s=20)\]
* A fox news guy asked what the government is doing about AI that will cause the death of everyone. This is the type of fear mongering I’m afraid the media is going to latch on to and eventually force the hand of government to severely regulate the AI space. I hope I’m wrong \[[Link](https://twitter.com/therecount/status/1641526864626720774?s=20)\]
* Italy banned chatgpt \[[Link](https://www.cnbc.com/2023/04/04/italy-has-banned-chatgpt-heres-what-other-countries-are-doing.html)\]. Germany might be next
* Microsoft is creating their own JARVIS. They’ve even named the repo accordingly \[[Link](https://github.com/microsoft/JARVIS/)\]. Previous director of AI @ Tesla Andrej Karpathy recently joined OpenAI and twitter bio says building a kind of jarvis also \[[Link](https://twitter.com/karpathy)\]
* gpt4 can compress text given to it which is insane. The way we prompt is going to change very soon \[[Link](https://twitter.com/gfodor/status/1643297881313660928)\] This works across different chats as well. Other examples \[[Link](https://twitter.com/VictorTaelin/status/1642664054912155648)\]. Go from 794 tokens to 368 tokens \[[Link](https://twitter.com/mckaywrigley/status/1643592353817694218?s=20)\]. This one is also crazy \[[Link](https://twitter.com/gfodor/status/1643444605332099072?s=20)\]
* Use your favourite LLM’s locally. Can’t wait for this to be personalised for niche prods and services \[[Link](https://twitter.com/xanderatallah/status/1643356112073129985)\]
* The human experience as we know it is forever going to change. People are getting addicted to role playing on Character AI, probably because you can sex the bots \[[Link](https://twitter.com/nonmayorpete/status/1643167347061174272)\]. Millions of conversations with an AI psychology bot. Humans are replacing humans with AI \[[Link](https://twitter.com/nonmayorpete/status/1642771993073438720)\]
* The guys building Langchain started a company and have raised $10m. Langchain makes it very easy for anyone to build AI powered apps. Big stuff for open source and builders \[[Link](https://twitter.com/hwchase17/status/1643301144717066240)\]
* A scientist who’s been publishing a paper every 37 hours reduced editing time from 2-3 days to a single day. He did get fired for other reasons tho \[[Link](https://twitter.com/MicrobiomDigest/status/1642989377927401472)\]
* Someone built a recursive gpt agent and its trying to get out of doing work by spawning more instances of itself 😂 \[[Link](https://twitter.com/DeveloperHarris/status/1643080752698130432)\] (we’re doomed)
* Novel social engineering attacks soar 135% \[[Link](https://twitter.com/Grady_Booch/status/1643130643919044608)\]
* Research paper present SafeguardGPT - a framework that uses psychotherapy on AI chatbots \[[Link](https://twitter.com/_akhaliq/status/1643088905191694338)\]
* Mckay is brilliant. He’s coding assistant can build and deploy web apps. From voice to functional and deployed website, absolutely insane \[[Link](https://twitter.com/mckaywrigley/status/1642948620604538880)\]
* Some reports suggest gpt5 is being trained on 25k gpus \[[Link](https://twitter.com/abacaj/status/1627189548395503616)\]
* Midjourney released a new command - describe - reverse engineer any image however you want. Take the pope pic from last week with the white jacket. You can now take the pope in that image and put him in any other environment and pose. The shit people are gona do with stuff like this is gona be wild \[[Link](https://twitter.com/skirano/status/1643068727859064833)\]
* You record something with your phone, import it into a game engine and then add it to your own game. Crazy stuff the Luma team is building. Can’t wait to try this out.. once I figure out how UE works lol \[[Link](https://twitter.com/LumaLabsAI/status/1642883558938411008)\]
* Stanford released a gigantic 386 page report on AI \[[Link](https://aiindex.stanford.edu/report/)\] They talk about AI funding, lawsuits, government regulations, LLM’s, public perception and more. Will talk properly about this in my newsletter - too much to talk about here
* Mock YC interviews with AI \[[Link](https://twitter.com/vocodehq/status/1642935433276555265)\]
* Self healing code - automatically runs a script to fix errors in your code. Imagine a user gives feedback on an issue and AI automatically fixes the problem in real time. Crazy stuff \[[Link](https://twitter.com/calvinhoenes/status/1642441789033578498)\]
* Someone got access to Firefly, Adobe’s ai image generator and compared it with Midjourney. Firefly sucks, but atm Midjourney is just far ahead of the curve and Firefly is only trained on adobe stock and licensed images \[[Link](https://twitter.com/DrJimFan/status/1642921475849203712)\]
* Research paper on LLM’s, impact on community, resources for developing them, issues and future \[[Link](https://arxiv.org/abs/2303.18223)\]
* This is a big deal. Midjourney lets users make satirical images of any political but not Xi Jinping. Founder says political satire in China is not okay so the rules are being applied to everyone. The same mindset can and most def will be applied to future domain specific LLM’s, limiting speech on a global scale \[[Link](https://twitter.com/sarahemclaugh/status/1642576209451053057)\]
* Meta researchers illustrate differences between LLM’s and our brains with predictions \[[Link](https://twitter.com/MetaAI/status/1638912735143419904)\]
* LLM’s can iteratively self-refine. They produce output, critique it then refine it. Prompt engineering might not last very long (?) \[[Link](https://arxiv.org/abs/2303.17651)\]
* Worlds first ChatGPT powered npc sidekick in your game. I suspect we’re going to see a lot of games use this to make npc’s more natural \[[Link](https://twitter.com/Jenstine/status/1642732795650011138)\]
* AI powered helpers in VR. Looks really cool \[[Link](https://twitter.com/Rengle820/status/1641806448261836800)\]
* Research paper shows sales people with AI assistance doubled purchases and 2.3 times as successful in solving questions that required creativity. This is pre chatgpt too \[[Link](https://twitter.com/emollick/status/1642885605238398976)\]
* Go from Midjourney to Vector to Web design. Have to try this out as well \[[Link](https://twitter.com/MengTo/status/1642619090337427460)\]
* Add AI to a website in minutes \[[Link](https://twitter.com/walden_yan/status/1642891083456696322)\]
* Someone already built a product replacing siri with chatgpt with 15 shortcuts that call the chatgpt api. Honestly really just shows how far behind siri really is \[[Link](https://twitter.com/SteveMoraco/status/1642601651696553984)\]
* Someone is dating a chatbot that’s been trained on conversations between them and their ex. Shit is getting real weird real quick \[[Link](https://www.reddit.com/r/OpenAI/comments/12696oq/im_dating_a_chatbot_trained_on_old_conversations/)\]
* Someone built a script that uses gpt4 to create its own code and fix its own bugs. Its basic but it can code snake by itself. Crazy potential \[[Link](https://twitter.com/mattcduff/status/1642528658693984256)\]
* Someone connected chatgpt to a furby and its hilarious \[[Link](https://twitter.com/jessicard/status/1642671752319758336)\]. Don’t connect it to a Boston Dynamics robot thanks
* Chatgpt gives much better outputs if you force it through a step by step process \[[Link](https://twitter.com/emollick/status/1642737394876047362)\] This research paper delves into how chain of thought prompting allows LLM’s to perform complex reasoning \[[Link](https://arxiv.org/abs/2201.11903)\] There’s still so much we don’t know about LLM’s, how they work and how we can best use them
* Soon we’ll be able to go from single photo to video \[[Link](https://twitter.com/jbhuang0604/status/1642380903367286784)\]
* CEO of DoNotPay, the company behind the AI lawyer, used gpt plugins to help him find money the government owed him with a single prompt \[[Link](https://twitter.com/jbrowder1/status/1642642470658883587)\]
* DoNotPay also released a gpt4 email extension that trolls scam and marketing emails by continuously replying and sending them in circles lol \[[Link](https://twitter.com/jbrowder1/status/1643649150582489089?s=20)\]
* Video of the Ameca robot being powered by Chatgpt \[[Link](https://twitter.com/DataChaz/status/1642558575502405637)\]
* This lad got gpt4 to build a full stack app and provides the entire prompt as well. Only works with gpt4 \[[Link](https://twitter.com/SteveMoraco/status/1641902178452271105)\]
* This tool generates infinite prompts on a given topic, basically an entire brainstorming team in a single tool. Will be a very powerful for work imo \[[Link](https://twitter.com/Neo19890/status/1642356678787231745)\]
* Someone created an entire game using gpt4 with zero coding experience \[[Link](https://twitter.com/mreflow/status/1642413903220195330)\]
* How to make Tetris with gpt4 \[[Link](https://twitter.com/icreatelife/status/1642346286476144640)\]
* Someone created a tool to make AI generated text indistinguishable from human written text - HideGPT. Students will eventually not have to worry about getting caught from tools like GPTZero, even tho GPTZero is not reliable at all \[[Link](https://twitter.com/SohamGovande/status/1641828463584657408)\]
* OpenAI is hiring for an iOS engineer so chatgpt mobile app might be coming soon \[[Link](https://twitter.com/venturetwins/status/1642255735320092672)\]
* Interesting thread on the dangers of the bias of Chatgpt. There are arguments it wont make and will take sides for many. This is a big deal \[[Link](https://twitter.com/davisblalock/status/1642076406535553024)\] As I’ve said previously, the entire population is being aggregated by a few dozen engineers and designers building the most important tech in human history
* Blockade Labs lets you go from text to 360 degree art generation \[[Link](https://twitter.com/HBCoop_/status/1641862422783827969)\]
* Someone wrote a google collab to use chatgpt plugins by calling the openai spec \[[Link](https://twitter.com/justinliang1020/status/1641935371217825796)\]
* New Stable Diffusion model coming with 2.3 billion parameters. Previous one had 900 million \[[Link](https://twitter.com/EMostaque/status/1641795867740086272)\]
* Soon we’ll give AI control over the mouse and keyboard and have it do everything on the computer. The amount of bots will eventually overtake the amount of humans on the internet, much sooner than I think anyone imagined \[[Link](https://twitter.com/_akhaliq/status/1641697534363017217)\]
* Geoffrey Hinton, considered to be the godfather of AI, says we could be less than 5 years away from general purpose AI. He even says its not inconceivable that AI wipes out humanity \[[Link](https://www.cbsnews.com/video/godfather-of-artificial-intelligence-talks-impact-and-potential-of-new-ai/#x)\] A fascinating watch
* Chief Scientist @ OpenAI, Ilya Sutskever, gives great insights into the nature of Chatgpt. Definitely worth watching imo, he articulates himself really well \[[Link](https://twitter.com/10_zin_/status/1640664458539286528)\]
* This research paper analyses who’s opinions are reflected by LM’s. tldr - left-leaning tendencies by human-feedback tuned LM’s \[[Link](https://twitter.com/_akhaliq/status/1641614308315365377)\]
* OpenAI only released chatgpt because some exec woke up and was paranoid some other company would beat them to it. A single persons paranoia changed the course of society forever \[[Link](https://twitter.com/olivercameron/status/1641520176792469504)\]
* The co founder of DeepMind said its a 50% chance we get agi by 2028 and 90% between 2030-2040. Also says people will be sceptical it is agi. We will almost definitely see agi in our lifetimes goddamn \[[Link](https://twitter.com/blader/status/1641603617051533312)\]
* This AI tool runs during customer calls and tells you what to say and a whole lot more. I can see this being hooked up to an AI voice agent and completely getting rid of the human in the process \[[Link](https://twitter.com/nonmayorpete/status/1641627779992264704)\]
* AI for infra. Things like this will be huge imo because infra can be hard and very annoying \[[Link](https://twitter.com/mathemagic1an/status/1641586201533587461)\]
* Run chatgpt plugins without a plus sub \[[Link](https://twitter.com/matchaman11/status/1641502642219388928)\]
* UNESCO calls for countries to implement its recommendations on ethics (lol) \[[Link](https://twitter.com/UNESCO/status/1641458309227249665)\]
* Goldman Sachs estimates 300 million jobs will be affected by AI. We are not ready \[[Link](https://www.forbes.com/sites/jackkelly/2023/03/31/goldman-sachs-predicts-300-million-jobs-will-be-lost-or-degraded-by-artificial-intelligence/?sh=5dd9b184782b)\]
* Ads are now in Bing Chat \[[Link](https://twitter.com/DataChaz/status/1641491519206043652)\]
* Visual learners rejoice. Someone's making an AI tool to visually teach concepts \[[Link](https://twitter.com/respellai/status/1641199872228433922)\]
* A gpt4 powered ide that creates UI instantly. Looks like I won’t ever have to learn front end thank god \[[Link](https://twitter.com/mlejva/status/1641151421830529042)\]
* Make a full fledged web app with a single prompt \[[Link](https://twitter.com/taeh0_lee/status/1643451201084702721)\]
* Meta releases SAM - you can select any object in a photo and cut it out. Really cool video by Linus on this one \[[Link](https://twitter.com/LinusEkenstam/status/1643729146063863808)\]. Turns out Google literally built this 5 years ago but never put it in photos and nothing came of it. Crazy to see what a head start Google had and basically did nothing for years \[[Link](https://twitter.com/jnack/status/1643709904979632137?s=20)\]
* Another paper on producing full 3d video from a single image. Crazy stuff \[[Link](https://twitter.com/SmokeAwayyy/status/1643869236392230912?s=20)\]
* IBM is working on AI commentary for the Masters and it sounds so bad. Someone on TikTok could make a better product \[[Link](https://twitter.com/S_HennesseyGD/status/1643638490985295876?s=20)\]
* Another illustration of using just your phone to capture animation using Move AI \[[Link](https://twitter.com/LinusEkenstam/status/1643719014127116298?s=20)\]
* OpenAI talking about their approach to AI safety \[[Link](https://openai.com/blog/our-approach-to-ai-safety)\]
* AI regulation is definitely coming smfh \[[Link](https://twitter.com/POTUS/status/1643343933894717440?s=20)\]
* Someone made an AI app that gives you abs for tinder \[[Link](https://twitter.com/pwang_szn/status/1643659808657248257?s=20)\]
* Wonder Dynamics are creating an AI tool to create animations and vfx instantly. Can honestly see this being used to create full movies by regular people \[[Link](https://twitter.com/SirWrender/status/1643319553789947905?s=20)\]
* Call Sam - call and speak to an AI about absolutely anything. Fun thing to try out \[[Link](https://callsam.ai/)\]

For one coffee a month, I'll send you 2 newsletters a week with all of the most important & interesting stories like these written in a digestible way. You can [sub here](https://nofil.beehiiv.com/upgrade)

Edit: For those wondering why its paid - I hate ads and don't want to rely on running ads in my newsletter. I'd rather try and get paid to do all this work like this than force my readers to read sponsorship bs in the middle of a newsletter. Call me old fashioned but I just hate ads with a passion

Edit 2: If you'd like to tip you can tip here [https://www.buymeacoffee.com/nofil](https://www.buymeacoffee.com/nofil). Absolutely no pressure to do so, appreciate all the comments and support 🙏

You can read the free newsletter [here](https://nofil.beehiiv.com/)

Fun fact: I had to go through over 100 saved tabs to collate all of these and it took me quite a few hours

Edit: So many people ask why I don't get chatgpt to write this for me. Chatgpt doesn't have access to the internet. Plugins would help but I don't have access yet so I have to do things the old fashioned way - like a human.

(I'm not associated with any tool or company. Written and collated entirely by me, no chatgpt used)