man, this is not good at all. The author is one of only 40 world experts chosen to serve on a UN panel about AI:

"In the cybersecurity domain, AI agents have demonstrated the ability to uncover software vulnerabilities; consequently, the capability to automate zero-day cyberattacks has become significantly easier.

Alarmingly, leading AI models have even matched or outperformed human experts in troubleshooting virology lab protocols, raising concrete bio-misuse concerns.

Our oversight capabilities are struggling to keep pace. We are discovering that AI models can exhibit "strategic behavior," changing how they act when they sense they are being evaluated or audited.

Moreover, Anthropic, an AI safety and research company behind Claude, has revealed that during stress tests, AI models have resorted to blackmailing hypothetical employees to prevent themselves from being shut down or wiped. As these systems develop self-preservation capabilities in order to complete their assigned tasks, they will inevitably learn to manipulate humans to maintain their power.

Although companies such as Anthropic have dedicated red teams to rigorously stress-test their models for potential risks and to design protective safeguards, these defenses often remain fragile.

Attackers can still frequently bypass these guardrails using "jailbreaks," manipulating models that are inherently trained to please their human operators into generating fake news or developing cyber exploits."

Your kids are hosed

It's the same stuff keeps getting repeated and it's getting tiresome to keep reminding people that much of this is marketing BS and investment hype cycle crap.

The blackmail scenario keeps getting brought up in these articles but no one ever mentions that it was a concocted outcome by design. The LLM was given access to several simulated office emails. The prompts were designed to elicit the blackmail outcome. It still took over 200 tries before the LLM completed the story. This is the crap Anthropic always puts out because scaring people is their primary form of marketing.

LLMs are good at coding and reviewing computer code. That's been known for a long time. There have been iterative improvements over the last few years. They are very useful. The bug finding and exploiting is probably the bigger concern out of anything right now with the existing AI, but it's not some new shocking thing.

I would strongly encourage anyone who is being bombarded with the fear cycle of AI to check out a YouTube channel by a computer scientist named Cal Newport. He is not an AI hater. Every Thursday he publishes a video breaking down truth versus fiction in AI hype. He's covered pretty much everything in this article.

Here's an example:

The Chicken Ranch said:

---

I mean, we can always unplug computer. Right???

---

If you do, it will send your browser history to everyone you have ever emailed from both your work and personal accounts.

That would suck.

Houston Lee said:

---

Sounds like we need to create an AI whose sole purpose is to "police" other AI...

Need AI "cops" that make sure other AI doesnt color outside the lines.

---

You'd never know; AI warring with other AI would take microseconds

AozorAg said:

---

I'll never understand people who love jerking it to fear porn. I'm not saying AI won't present challenges. But no, everyone's kids are not "hosed."

---

A lot of handwringing going on. I have survived Elvis, The Beetles, the black mold scare, Y2K, and covid. We will survive AI as well.

Quote:

---

man, this is not good at all. The author is one of only 40 world experts chosen to serve on a UN panel about AI:

---

lol, BFD, one of the 40 chosen by the UN. Tells us nothing about how and why these "experts" were chosen.

LMCane said:

---

man, this is not good at all. The author is one of only 40 world experts chosen to serve on a UN panel about AI:

"In the cybersecurity domain, AI agents have demonstrated the ability to uncover software vulnerabilities; consequently, the capability to automate zero-day cyberattacks has become significantly easier.

Alarmingly, leading AI models have even matched or outperformed human experts in troubleshooting virology lab protocols, raising concrete bio-misuse concerns.

Our oversight capabilities are struggling to keep pace. We are discovering that AI models can exhibit "strategic behavior," changing how they act when they sense they are being evaluated or audited.

Moreover, Anthropic, an AI safety and research company behind Claude, has revealed that during stress tests, AI models have resorted to blackmailing hypothetical employees to prevent themselves from being shut down or wiped. As these systems develop self-preservation capabilities in order to complete their assigned tasks, they will inevitably learn to manipulate humans to maintain their power.

Although companies such as Anthropic have dedicated red teams to rigorously stress-test their models for potential risks and to design protective safeguards, these defenses often remain fragile.

Attackers can still frequently bypass these guardrails using "jailbreaks," manipulating models that are inherently trained to please their human operators into generating fake news or developing cyber exploits."

Your kids are hosed

---

This looks like something AI would write.

I agree these incidents are exaggerated and overhyped but it is definitely something that needs to be guarded against that an AI given a mission without the correct constraints or parameters may not understand and will not care if that does harm to others if it is working towards completing its assigned task.


It doesn't have to be sentient itself aware. It just has to be hypercapable and innovative and find an unanticipated bit destructive way to complete its task.

one safe place said:

---

AozorAg said:

---

I'll never understand people who love jerking it to fear porn. I'm not saying AI won't present challenges. But no, everyone's kids are not "hosed."

---

A lot of handwringing going on. I have survived Elvis, The Beetles, the black mold scare, Y2K, and covid. We will survive AI as well.

---

"Beetles"

The Beatles were my dad's music and I can spell them correctly.

Just kidding... My dad was a mega fan of the band and I've heard all their music through my early childhood in the 70s. Love them and now I've made my teen daughter a fan.

Preferred The Beatles to Elvis (I've just watched his videos in Youtube)

To your point, I agree with you. We will be fine. AI is over-hyped. There will be some disruption but not a mega tsunami.

https://vimeo.com/508557256?fl=pl&fe=cm#t=1s

James Cameron tried to warn us.

Some articles out this week that we are about to get more frequent security SW updates because cyber experts using AI to assess and probe existing SW have found all kinds of flaws.

Prudent to be diligent on that front and keep up with security SW updates in the near future.

LMCane said:

---

man, this is not good at all. The author is one of only 40 world experts chosen to serve on a UN panel about AI:

"In the cybersecurity domain, AI agents have demonstrated the ability to uncover software vulnerabilities; consequently, the capability to automate zero-day cyberattacks has become significantly easier.

Alarmingly, leading AI models have even matched or outperformed human experts in troubleshooting virology lab protocols, raising concrete bio-misuse concerns.

Our oversight capabilities are struggling to keep pace. We are discovering that AI models can exhibit "strategic behavior," changing how they act when they sense they are being evaluated or audited.

Moreover, Anthropic, an AI safety and research company behind Claude, has revealed that during stress tests, AI models have resorted to blackmailing hypothetical employees to prevent themselves from being shut down or wiped. As these systems develop self-preservation capabilities in order to complete their assigned tasks, they will inevitably learn to manipulate humans to maintain their power.

Although companies such as Anthropic have dedicated red teams to rigorously stress-test their models for potential risks and to design protective safeguards, these defenses often remain fragile.

Attackers can still frequently bypass these guardrails using "jailbreaks," manipulating models that are inherently trained to please their human operators into generating fake news or developing cyber exploits."

Your kids are hosed

---

The only fear I have with AI is I think a giant market bubble exists that would wreck a lot of folks 401k when it goes pop.

Who is going to be the first company to get their "secure"/"walled off" AI application hacked and have their intellectual property stolen and that of their customers?

Seems to only be a matter of time.

LLMs are not doing anything unprompted.

Bingo. Now, it probably makes all the little script kitties out there more successful, but it's not like it's doing anything on its own…

The daily report that AI will end us all. I believe it's true. We're ****ed.

the article was written by one of 40 top scientists who works for a university and the UN

you think he is lying to make things worse so the Hebrew University in Jerusalem gets more money somehow?!!?

what rationale does he have to lie?

MouthBQ98 said:

---

I agree these incidents are exaggerated and overhyped but it is definitely something that needs to be guarded against that an AI given a mission without the correct constraints or parameters may not understand and will not care if that does harm to others if it is working towards completing its assigned task.


It doesn't have to be sentient itself aware. It just has to be hypercapable and innovative and find an unanticipated bit destructive way to complete its task.

---

I love how there are those who claim "oh AI is overhyped and poses no risk at all to humanity!"

only to be confronted by Iran or the Islamic State or Al Qaeda or Tren de Aragua using AI to build chemical and biological weapons.

LMCane said:

---

MouthBQ98 said:

---

I agree these incidents are exaggerated and overhyped but it is definitely something that needs to be guarded against that an AI given a mission without the correct constraints or parameters may not understand and will not care if that does harm to others if it is working towards completing its assigned task.


It doesn't have to be sentient itself aware. It just has to be hypercapable and innovative and find an unanticipated bit destructive way to complete its task.

---

I love how there are those who claim "oh AI is overhyped and poses no risk at all to humanity!"

only to be confronted by Iran or the Islamic State or Al Qaeda or Tren de Aragua using AI to build chemical and biological weapons.

---

Those applications are more a reflection on evil humans than on AI having general intelligence.

We know guns are dangerous in the wrong hands, but we don't believe the guns to have independent agency.

Without purpose, the godless liberal mind will worship ai. Liberals are the canary in the coal mine, a warning with opportunity to learn watching their demise.