Latest / Tech Talks With Kinsoft / Last Week in Tech – Anthropic's Exploit-Writing Model, Meta Goes Closed-Source, and a Critical Notebook RCE

Transcript

Last Week in Tech – Anthropic's Exploit-Writing Model, Meta Goes Closed-Source, and a Critical Notebook RCE · Tech Talks With Kinsoft

• 0:00
  Picture your digital life right now. You know,
• 0:03
  like every app you use, the corporate databases
• 0:05
  you access, even the cloud storage holding years
• 0:08
  of your work. Yeah, pretty much your entire digital
• 0:11
  footprint. Right, exactly. Up until recently,
• 0:13
  we basically relied on human beings to build
• 0:16
  the invisible walls, protecting all that data.
• 0:19
  And we relied on humans to rush in and fix them
• 0:20
  when a crack appeared, too. Yeah, but a new tech
• 0:23
  update crossing our desk from Kinsoft, along
• 0:26
  with this... honestly shocking internal test
• 0:29
  from the AI giant Anthropic Well, it basically
• 0:32
  proves that era is over. It really is over. So
• 0:35
  today's deep dive is all about the machine speed
• 0:37
  threat landscape. We're looking at why human
• 0:40
  speed IT responses are officially dead and why
• 0:43
  a 10 -hour window is now essentially a worst
• 0:45
  -case scenario. So, okay, let's unpack this.
• 0:48
  Yeah, because the stakes here, I mean, they go
• 0:50
  far beyond just a standard software update or
• 0:52
  a routine patch. The Kinsarf briefing and the
• 0:55
  anthropic data we have today represent a fundamental
• 0:57
  shift in how the global infrastructure defends
• 0:59
  its data. A total paradigm shift. Completely.
• 1:02
  We are moving away from a reactive, human -driven
• 1:05
  security model right into an automated machine
• 1:09
  speed arms race. Well, let's start with what
• 1:11
  I can only really describe as an absolute bombshell
• 1:14
  from Anthropic. They unveiled the details of
• 1:17
  a program called Project Glasswing. Ah, yeah,
• 1:20
  the internal tests. Right. And now this was designed
• 1:23
  from the ground up to be a defensive -only initiative.
• 1:26
  They took an unreleased frontier AI. So like
• 1:30
  the absolute bleeding edge of their technology
• 1:32
  and granted early access to about 50 security
• 1:35
  partners. Which, you know, is a pretty standard
• 1:37
  industry practice, historically speaking. Really?
• 1:39
  Yeah. I mean, you give your defenders the most
• 1:41
  powerful lens possible to inspect the code, right?
• 1:44
  Hoping they find the cracks before anyone else
• 1:46
  does. Well, the testing phase produced results
• 1:47
  that completely upended that whole plan. Because
• 1:50
  during this closed test, that unreleased AI model
• 1:53
  autonomously produced 181 working exploits for
• 1:57
  Firefox. Oh, wow. And that's not a typo. 181.
• 2:01
  And Firefox isn't some beta app coded over a
• 2:04
  weekend by a college student. No, not at all.
• 2:06
  It's a mature, heavily scrutinized web browser
• 2:09
  that's been around for decades. Exactly. Finding
• 2:12
  a single working exploit in a program like that
• 2:15
  usually takes a dedicated human researcher. weeks,
• 2:19
  if not months, of painstaking analysis. And Anthropic's
• 2:22
  model just spun up 181 of them autonomously.
• 2:26
  Completely autonomously. It was so intense that
• 2:29
  they actually judged the model to be too dangerous
• 2:32
  for broad release. You know, what's fascinating
• 2:34
  here is the underlying mechanics of the dual
• 2:37
  use dilemma in artificial intelligence. Yeah,
• 2:39
  dual use dilemma. Right. Because the exact same
• 2:41
  computational power and deep pattern recognition
• 2:44
  required to defend a complex system at scale
• 2:47
  is precisely what makes the AI a devastatingly
• 2:50
  fast attacker. That makes sense. Yeah, because
• 2:53
  to patch a vulnerability, an AI first has to
• 2:56
  deeply understand how to break the software,
• 2:58
  right? It has to ingest millions of lines of
• 3:00
  code, spot the logical flaw, and understand precisely
• 3:03
  how an unexpected input could, say, cause the
• 3:06
  system to behave abnormally or leak memory. Right,
• 3:09
  because it doesn't read code line by line like
• 3:11
  a human trying to read a novel. It reads it structurally.
• 3:14
  It maps the entire architecture simultaneously.
• 3:17
  So it spots a variable input over here and immediately
• 3:20
  calculates how it might overflow a buffer like
• 3:23
  10 modules away. And if you ask it to generate
• 3:26
  a patch, it uses that architectural map to block
• 3:29
  the bad input. Okay. But if you tweak the prompt
• 3:32
  slightly or, you know, if the model is operating
• 3:34
  autonomously without a strict ethical governor,
• 3:36
  it uses that exact same structural understanding
• 3:39
  to generate the attack payload. So it's two sides
• 3:42
  of the same coin. Exactly. The capability to
• 3:45
  secure and the capability to destroy are structurally
• 3:48
  identical in these models. Because it is a machine,
• 3:52
  it analyzes a billion lines of code in the time
• 3:54
  it takes a human analyst to basically open their
• 3:57
  laptop. So it's like inventing a highly advanced
• 3:59
  lock -picking robot to test the security of bank
• 4:03
  vaults, only to realize your robot is so efficient
• 4:05
  it could autonomously empty every bank in the
• 4:08
  city before lunch. That's a perfect way to put
• 4:10
  it. You can't build a robot that only knows how
• 4:12
  to pick the lock. for the bank manager. If it
• 4:14
  knows how the pins align, it knows how to defeat
• 4:17
  them. And this is exactly why Anthropic had to
• 4:20
  slam the brakes on releasing Project Glasswing.
• 4:23
  Because the risk was just too high. Yeah, the
• 4:25
  defensive benefits were enormous, sure. But dropping
• 4:29
  that uninhibited capability into the open internet,
• 4:32
  it was deemed catastrophic. Which really forces
• 4:35
  us to look at the internet you and I are using
• 4:37
  right now. Like we can theorize about unreleased
• 4:40
  frontier models in a lab all day, but the Kinsoft
• 4:43
  brief grounds this in current reality. It really
• 4:46
  does. We know AI can attack at machine speed.
• 4:48
  So how fast are the criminals actually moving
• 4:52
  today against the systems we currently rely on?
• 4:55
  Well, the Kinsoft brief highlights a really critical
• 4:57
  flaw discovered in a popular open source Python
• 5:00
  notebook tool called Merimo. OK, Merimo. Yeah.
• 5:03
  Tools like Merimo are foundational in data science,
• 5:06
  AI development, corporate analytics. And the
• 5:08
  flaw they found was a pre -authentication remote
• 5:11
  code execution vulnerability. Pre -auth RCE.
• 5:15
  Right. A pre -AUC RCE. So just to be clear, we
• 5:18
  aren't talking about phishing emails or tricking
• 5:20
  an employee into clicking a malicious link here.
• 5:22
  No, not at all. This is a zero interaction takeover.
• 5:26
  If the software is running on a server, a hacker
• 5:29
  can basically reach across the internet, send
• 5:31
  a specific command, and completely own the machine.
• 5:34
  They can run whatever code they want without
• 5:36
  ever needing a password. Exactly. A systemic
• 5:39
  vulnerability like this is like... It's like
• 5:42
  a building's intercom system being wired directly
• 5:44
  into the elevator controls. Oh, wow. Yeah. Anyone
• 5:47
  on the street can just press a button and send
• 5:49
  the elevator straight to the penthouse without
• 5:51
  ever stepping foot inside or scanning a security
• 5:54
  badge. And here is the metric from the brief
• 5:56
  that should send a chill down the spine of literally
• 6:01
  every IT professional listening. Yeah, this is
• 6:04
  the scary part. From the moment that flaw in
• 6:07
  Murumo was publicly disclosed, it took only about
• 6:09
  10 hours for it to be exploited in the wild to
• 6:12
  drop malware. Just 10 hours. 10 hours from public
• 6:16
  disclosure to global infection. I mean, 10 hours
• 6:18
  is a blink of an eye in corporate IT infrastructure.
• 6:21
  Okay, so I get the 10 -hour panic, but wait.
• 6:23
  I had to play devil's advocate for a second.
• 6:25
  Sure, go ahead. 10 hours sounds fast, but I mean,
• 6:27
  isn't that basically a full workday? If a flaw
• 6:30
  drops at 8 a .m., why can't a company just patch
• 6:32
  it by 5 p .m. before they go home? Plus, aren't
• 6:35
  companies using AI to automate the patching process
• 6:38
  too? Like if the attackers have bots scanning
• 6:41
  for vulnerabilities and launching exploits in
• 6:43
  10 hours, why don't defenders have bots automatically
• 6:47
  applying the patches? Why are we still relying
• 6:49
  on human bureaucracy? It sounds simple, right?
• 6:52
  But it really comes down to system fragility
• 6:54
  and the burden of consequence. Burden of consequence.
• 6:57
  Yeah. See, an attacker only has to be right once
• 7:00
  to compromise a system. A defender has to ensure
• 7:03
  that applying a patch doesn't accidentally bring
• 7:05
  down the entire company. If you rush an automated
• 7:09
  patch on a core data tool like Merimo without
• 7:12
  human oversight, it might inadvertently change,
• 7:15
  say, how a Python script parses a timestamp.
• 7:18
  Oh, which breaks something down the line. Exactly.
• 7:21
  Suddenly, your logistics database rejects every
• 7:23
  incoming shipping order because date format is
• 7:26
  slightly off. So the cure just halted your global
• 7:29
  supply chain. That's how a well -intentioned
• 7:31
  automated patch causes far more financial damage
• 7:35
  than the hacker ever could. So defenders are
• 7:37
  basically forced to test everything manually.
• 7:40
  Yeah. Which creates a massive bottleneck. Exactly.
• 7:42
  Let's actually walk through those 10 hours. In
• 7:45
  hour one, the vulnerability is disclosed on the
• 7:48
  security forum. The IT team have to actually
• 7:51
  see the notification. thousands of other daily
• 7:55
  alerts. Right. And let's assume they catch it
• 7:57
  instantly. Hours two and three are spent just
• 7:59
  figuring out if the company even uses Marimo,
• 8:02
  where it's installed and what systems depend
• 8:04
  on it. Ah, shadow IT. Yeah. Which is really just
• 8:07
  a polite way of saying the marketing department
• 8:09
  bought a random software license on a corporate
• 8:11
  credit card and didn't bother telling the security
• 8:14
  team. Yeah. Shadow IT makes asset discovery an
• 8:16
  absolute nightmare. Security teams are hunting
• 8:19
  for software they don't even know they own. That's
• 8:21
  terrifying. It is. By hour four. you've hopefully
• 8:24
  tracked down the vulnerable servers. Then hours
• 8:27
  five through eight are spent in a staging environment,
• 8:29
  desperately testing that patch so your logistics
• 8:32
  database doesn't crash. Right. You're constantly
• 8:35
  balancing the demands of the security team who
• 8:37
  want the patch applied immediately and the ops
• 8:40
  team who demand 100 % uptime. And while your
• 8:43
  IT team is doing all this careful human speed
• 8:45
  work, the attackers are entirely unburdened by
• 8:49
  QA testing or operational uptime. Right. They
• 8:51
  don't care about your uptime. Exactly. The second
• 8:54
  the exploit went public, criminal syndicates
• 8:56
  fed it straight into their automated scanners.
• 8:58
  And those bots scan millions of IP addresses
• 9:01
  a second. Yeah. When they find an unpatched Marimo
• 9:03
  instance, the malware is dropped instantly. There
• 9:06
  are no staging environments, no approval meetings,
• 9:09
  just pure machine speed execution. The takeaway
• 9:12
  from the Kinsoft source here is absolute. If
• 9:14
  your teams run open source data tools, they must
• 9:17
  be on an incredibly fast patch cycle. The gap
• 9:20
  between public disclosure and criminal exploitation,
• 9:22
  it went from months to weeks to days. And now
• 9:26
  we are at 10 hours. Yeah. And in the very near
• 9:28
  future, it will be 10 minutes. Which brings up
• 9:31
  a massive ripple effect across the whole tech
• 9:33
  industry. With open source tools like Marimo
• 9:36
  proving so vulnerable to rapid exploitation and
• 9:39
  frontier models like Anthropix proving so dangerous
• 9:42
  if they fall into the wrong hands. Well, the
• 9:44
  biggest tech giants on the planet are abruptly
• 9:47
  changing their business strategies. Massive shifts.
• 9:50
  Yeah. Here's where it gets really interesting.
• 9:52
  Meta, you know, the company behind Facebook and
• 9:54
  Instagram, they just launched. MuseSpark. And
• 9:57
  MuseSpark is their very first proprietary closed
• 10:00
  AI model. Which is a complete reversal of their
• 10:03
  established operational strategy. Totally. Meta
• 10:06
  has historically been the ultimate champion of
• 10:08
  the open source AI movement. They built their
• 10:11
  entire AI reputation on the Lama models, making
• 10:14
  them freely available for developers to download,
• 10:16
  to modify, and build upon. Right. They pushed
• 10:19
  to democratize AI. And practically overnight,
• 10:22
  they are retreating into a closed, proprietary
• 10:24
  walled garden with MuseSpark. They are locking
• 10:27
  the doors. They aren't the only ones making big
• 10:30
  moves. The Kinsoft update also highlights some
• 10:32
  enormous financial numbers flashing from Amazon
• 10:35
  and OpenAI. Specifically, AWS Amazon Web Services
• 10:40
  noted their AI business has just passed a massive
• 10:44
  $15 billion run rate. A $15 billion run rate,
• 10:48
  meaning based on their current explosive monthly
• 10:50
  revenue, they are pacing to make $15 billion.
• 10:53
  billion this year purely on AI infrastructure.
• 10:57
  Right. And when AWS is projecting $15 billion
• 11:00
  in AI revenue, it proves that enterprise companies
• 11:03
  are completely intertwining their core logistics,
• 11:05
  financials, and customer data with artificial
• 11:08
  intelligence. They're slowly bought in. Absolutely.
• 11:10
  And Meta sees that corporate money on the table.
• 11:13
  They know enterprise clients like banks, hospitals,
• 11:15
  energy grids. They won't buy into an open source
• 11:17
  model where structural vulnerabilities are broadcast
• 11:20
  to hackers the exact moment they're found. Right.
• 11:22
  Meta closed their model because enterprise security
• 11:24
  and the billions of dollars attached to it absolutely
• 11:27
  demands it. So how does a closed proprietary
• 11:29
  model technically mitigate that 10 hour vulnerability
• 11:33
  window compared to an open source one? Like functionally.
• 11:38
  Well, think about the mechanics of visibility.
• 11:40
  Open source collaboration is wonderful for innovation
• 11:43
  because thousands of developers look at the code,
• 11:46
  improve it, find creative uses. But as we saw
• 11:49
  with Murimo, open source also means that when
• 11:51
  a structural flaw is found. Everyone sees the
• 11:54
  underlying code at exactly the same time. The
• 11:57
  criminals see the blueprints at the exact moment
• 11:59
  the defenders do. It's just an open book for
• 12:01
  anyone to read. Exactly. And when you introduce
• 12:04
  AI to that dynamic, the attackers can weaponize
• 12:07
  that open book in hours. Wow. So by pivoting
• 12:10
  to a closed model like MuseSpark, Meta is hiding
• 12:14
  the blueprints. If a vulnerability is discovered
• 12:16
  internally, they can patch it behind closed doors,
• 12:19
  push the update globally across their managed
• 12:21
  servers, and the public... including the attackers,
• 12:23
  never actually sees the raw mechanics of the
• 12:26
  flaw. Oh I see. It essentially removes the public
• 12:28
  race against the clock. That makes total sense.
• 12:31
  I mean, if an unreleased, closed model like Anthropix
• 12:34
  can autonomously generate 181 working exploits
• 12:38
  in a defensive test, giving bad actors open source
• 12:41
  access to a frontier model's underlying architecture
• 12:43
  is essentially handing them a machine gun in
• 12:46
  a glass house. Yeah, that's exactly what it is.
• 12:48
  Tech companies are being forced to prioritize
• 12:50
  heavily guarded security over collaboration.
• 12:52
  The financial stakes are just too high to leave
• 12:55
  the hood of the car open for anyone to tinker
• 12:57
  with the engine. A machine speed threat landscape
• 13:00
  basically forces you to remove human delay from
• 13:02
  the equation and tightly control the technology
• 13:05
  generating the threats. Walled gardens are kind
• 13:08
  of the only current viable response when your
• 13:11
  adversary operates in milliseconds. Okay. So
• 13:14
  to summarize everything we've pulled from these
• 13:15
  sources for you today, we are living in a brand
• 13:18
  new era. The anthropic test proved that artificial
• 13:21
  intelligence can discover and weaponize software
• 13:23
  vulnerabilities at a scale and speed humans simply
• 13:27
  cannot match. The Marimo incident proved that
• 13:29
  cyber criminals are already exploiting flaws
• 13:32
  in the wild within 10 hours of discovery. And
• 13:34
  the massive strategic shifts from companies like
• 13:36
  Meta and Amazon show the industry is reacting
• 13:39
  by prioritizing walled gardens and extreme security
• 13:42
  over open access. Because human monitoring is
• 13:45
  just no longer a sufficient shield for your data.
• 13:48
  The Kinsoft Tech Talks brief ends with a stark
• 13:51
  piece of advice. Stay patched. Stay skeptical.
• 13:54
  They note that listeners looking for help keeping
• 13:56
  their patching on pace with these incredibly
• 13:59
  fast attackers can visit www .kinsoft .com .au.
• 14:05
  Because if your patching isn't automated and
• 14:06
  immediate, you're hopelessly outgunned. It really
• 14:09
  makes you evaluate your own digital life, doesn't
• 14:12
  it? It really does. I'd ask you, the listener,
• 14:14
  to just reflect on your habits. When your phone
• 14:17
  says there's a critical security update, do you
• 14:19
  hit install now or do you hit remind me tomorrow
• 14:21
  because you're in the middle of a meeting? I
• 14:23
  do that all the time. We all do. But how quickly
• 14:26
  do you update your apps, your open source tools,
• 14:29
  or your operating systems? Remind me, tomorrow
• 14:31
  used to be harmless procrastination. But in a
• 14:34
  machine speed landscape, tomorrow is 14 hours
• 14:37
  too late. That is a sobering reality check. We
• 14:40
  are trusting these incredible AI tools to act
• 14:42
  as our ultimate shield, right? To patch the holes
• 14:45
  and monitor the walls faster than we ever could.
• 14:47
  And this raises an important question, something
• 14:48
  really profound to leave you with. What's that?
• 14:51
  Well... We know the dual use dilemma is real.
• 14:53
  AI can defend an attack with equal brilliance.
• 14:56
  If an AI can autonomously generate 181 working
• 15:00
  exploits in a controlled defensive test today,
• 15:03
  what happens tomorrow when someone designs an
• 15:06
  AI specifically to find the hidden flaws in the
• 15:08
  very AI models we are relying on to protect us?
• 15:11
  Wow. The ultimate lock picking robot designed
• 15:14
  solely to dismantle the security robot you just
• 15:16
  bought.