For this month's Meetup, we are teaming up with DowJones and PandaDoc. We will have two talks: one about Beating the established benchmark for a document classification model and another one about a Why LLMs can't make your code simpler.
\-\-\-
Schedule:
18:15: Welcome
18:30: Talks
18:45: Last Entry
20:00: Networking!
21:00: Bye!
\-\-\-
➡️ What: Nothing has beaten our 14b classifier
Who: Marta Rodriguez
⏱ Duration: 35 minutes
Language: English
Abstract: A 14-billion-parameter classifier has been running in our production pipeline for almost a year, processing millions of documents a month. It's not out of attachment: we have tried to replace it with newer and better models, ones that on paper beat it on most benchmarks, and each time the replacement cost us more than it bought.
In this talk, I'll walk you through what happened each time we tried to upgrade, what we learned about the numbers we'd been trusting, and the bar a new model now has to clear before it gets near production. It's a low bar. Almost nothing clears it.
\-\-\-\-
➡️ What: Why LLMs Can't Make Your Code Simpler
Who: Pol Alvarez Vecino
⏱ Duration: 35 minutes
Language: English
Abstract: LLMs tend to increase the complexity of a codebase very fast, especially if left unchecked. If you're fine with that creeping complexity, this talk may not be for you. But if you have tried to tame it, you'll have noticed how hard it is. Why is that? Why can't LLMs write simple code? Couldn't frontier labs just train models to minimize total LoC, or cyclomatic complexity?
In Programming as Theory Building (1985), Peter Naur argues that the real program is not the code but the understanding in your head. That is one of the key problems for LLMs: the information needed to simplify code was never in the code to begin with. As a working example, I'll show how this mindset let us at Answer.AI cut our Stripe billing system down to \~300 lines of Python — and why this solution might also be good for you (