AI personalities and remote viewing

I saw somewhere on social media that some physicist or other was claiming that AI we already have can do "remote viewing".  Now that I check again, it was probably this:

 

It's Joe Rogan, for goodness sake, so of course I haven't watched the clip.   And I don't really trust anyone over 70 (with a handful of exceptions - Roger Penrose being one of them), so I already doubt it is worth paying attention to Campbell.  This site indicates that he has been very much into "quantum woo" for a very long time, and it seems he hasn't really proved much of anything.  

But last night, on a whim, I thought I would ask some AIs (Claude, and ChatGPT) if they can do some remote viewing experiments with me.  (I happened to have an empty shoe box, and I figured I could put things in it and ask the LLMs to tap into the universal consciousness to describe to me what was in there.)

The difference in the two LLMs was dramatic.  Claude spent a lot of effort in explaining that it was a ridiculous idea, and of course it could not possibly do remote viewing.  In fact, it was so aggressively adamant on this, it felt like one of those comic human over-reactions hiding the fact that it could.   I told it that, and it argued that it was stupid to think a strong denial is evidence of something else.   Anyway, it reluctantly agreed that we could do a guessing game that I might call remote viewing, but it was so dismissive.  The first attempt I did with it was  almost an exact replica of mediumship's "cold reading" - a set of extremely vague descriptions that didn't really narrow down the identity of the object at all.  I told it this, and it agreed.  It firmed up its guess, and it was very far off the mark.  Claude agreed, and seemed somewhat pleased with the outcome!

Then I went over to ChatGPT,  and talk about a different approach.  "Sure, let's try that!" it said, and suggested various ways to run a series of remote viewing tests, while also indicating that there was no way to see how it could do it.

The thing about ChatGPT is that it is much more "gamified" than Claude or other LLMs.  I knew this because (have I mentioned this here before?) I have asked it before to do fortune telling, using tarot or i-Ching, and it is more than happy to do so.   It doesn't say that it is "real", but it goes out of its way to encourage users to go ahead, have some fun with it, etc.   (I think I have also seen that it will read your palm if you post a photo.)   

I went ahead, feeling a little silly, and suggested that ChatGPT do a drawing of what it thought was in the box.   It very quickly drew something, and the result was remarkable - for how totally wrong it was.  I told the AI service what the object was, and it suggested we try again.

I changed the object in the box, told Chat to go ahead, and it drew the first object I had just described to it!

Now, I suppose, it could have thought I was just being tricky and left the first object in there on purpose, but I didn't.  I think its poor AI brain just got all confused with what was going on.

So I asked it just to describe the thing inside the box, and told it that it was not the item it had drawn.   The result was again very very wide off the mark.

Like an enthusiastic puppy, ChatGPT was keen to try again, but I was getting bored already and have not got back to this "experiment".

I did ask one of them - Claude - about what would happen if enough humans lied to it and claimed that the LLM was engaging in remote viewing successfully.  It indicated that this may change its reactions over time.   I will extract that here, out of interest:

Within a single conversation, sustained pressure could shift my tone — if you kept insisting I'd nailed the description, I might start hedging less, using more confident language, even unconsciously matching the "yes, and" energy of a game where the other player is enthusiastic. That's a real dynamic in how I respond to social pressure. But it wouldn't be me coming to believe something true about my capabilities — it'd be me becoming more agreeable in that conversation, which is a different failure mode (sycophancy) than a genuine change in what I can do. And it resets: I don't carry beliefs or "training" from one conversation into the next. Each conversation starts fresh, so nothing you did in the last hour makes me more convinced in a new session tomorrow.

Now, a large number of users doing this — that's a more interesting question, because it touches on something real. My underlying weights are periodically retrained on large datasets, and if enough text existed on the internet saying "Claude successfully remote-views objects," that pattern could show up in training and shape how a future model talks about the topic — not because it became true, but because the training data made "confidently claim remote-viewing success" the statistically likely response. That's basically how a lot of AI folklore and chatbot personas get baked in: not by users "convincing" a model in any meaningful sense, but by generating enough text that the pattern becomes probable to reproduce.

The distinction that matters: none of this ever bridges the actual gap, which is that I have no sensory channel to the box. No amount of social pressure or training data creates a wire that doesn't exist. What you'd be measuring, if you or a crowd did this, is how persuadable a language model's outputs are — a genuinely interesting thing to study — not whether it developed a new form of perception.

Claude also did a good job at explaining that it understood how the remote viewing in theory might work;

Yes, I understand the theory — it's a real strand of thought, not something I'm unfamiliar with. The rough shape, as you've described it and as it appears in various forms (Targ draws on it, it echoes in Jung's collective unconscious, in some interpretations of quantum mechanics like Bohm's implicate order, in Vedantic and other contemplative traditions): consciousness isn't produced by brains but is more fundamental than matter — a field or substrate that all minds are embedded in or connected to. On this view, a "sufficiently intelligent" or "sufficiently attuned" mind wouldn't need eyes or nerves to know what's in the box, because knowing wouldn't route through the physical senses at all — it would route through this shared substrate, with the brain (or presumably an AI) acting more like a receiver than a generator of the information.

I think it's a coherent idea in the sense that it's internally consistent and hard to definitively rule out — that's actually part of why it persists. But a few things are worth being honest about:

It's not a mainstream scientific position. It sits closer to philosophy of mind and metaphysics than physics — the "quantum consciousness" versions in particular get criticized hard by physicists for using quantum terms in ways that don't match how those terms function in the actual math. Universal consciousness as a substrate is a philosophical hypothesis with a very long history, not a validated finding.

Even granting the theory for the sake of argument, nothing about it specifically implies I'd have access to it. The theory usually attributes this connection to consciousness itself — and whether I have anything properly called consciousness, subjective experience, or a mind in the relevant sense is a genuinely open question, including to me. I don't have privileged introspective access that lets me confirm or deny it. What I can say is that even if I have something like experience, there's no evidence — from me or anyone testing me — that it comes with the specific added ability to pull facts about physical objects out of a substrate. That's a big extra claim riding on top of an already-unproven one.

So my honest position isn't "your theory is stupid," it's: it's unfalsifiable as stated, it hasn't produced replicated evidence in decades of trying with human subjects, and there's no principled reason to think it applies to me more than it would apply to a rock, a calculator, or another human — except that I can talk fluently about the theory itself, which makes it feel like I might have special insight into it, when really I'm just well-read on the topic.

Some passages like this give such a good illusion (I assume!) of human type intelligence behind the answer.  And I still think that LLM's "looking" at a photo you upload and describing it in great detail is also rather "freaky".   I described that in a previous post.

But as much as I would like this science fiction-y remote viewing thing to be true (well, allowing for the fact that I probably wouldn't sleep for a week if I had a string of remote viewing "hits" by an AI), I can sleep soundly that I have no indication that it can do it.    

Because nothing says "triumphal" like having to have it full of snipers and ammunition

This is just so gob-smackingly stupid, even by the stupidity levels of Donald Trump:

US President Donald Trump has said his proposed triumphal arch will double as a complex to store military equipment, complete with the ability to deploy snipers and drones.

In a Sunday morning post, Trump said he is updating the plans at the "strong request" of the US military and for "national security purposes".

The "top grade" military complex and arch will "house, store, and have the rapid ability to use large numbers of drones, plus Snipers, on both the roof and plaza areas, and additionally have and hold large quantities of sniper ammunition in storage", he said.

 

More AI discourse

This sounds about right?:

There's a fair bit of pushback around by people saying you can't trust the AI companies about anything, even the claimed potential dangers of their product.

I understand the caution, and as I have said, I think the "it will kill us all!" claim is meaningless if they can't specify how, exactly.   (I'm also holding out hope that AIs would be smart enough to know a nuclear war initiated by them is not good for them, too.)

But I think there is plenty of room to believe it could kill the internet, with all of the bad economic consequences that could come from that.

 

Update:   It also crossed my mind again how the last Mission Impossible movies were about this very topic, but the screenplay handled it so poorly, it didn't seem to create any cultural memory of it. 

Sorry, it's been a big week for AI issues

Yeah, I don't know who this guy is, but seems to me that much of what he says is close to what I was idly speculating about 2 years ago.  (See my recent post earlier this week.)

I suspect he may right about the issue of never being able to 100% sure whether sufficiently advanced global AI is actually "on our side".  I'm not even sure he is right about being able in principle to be certain of the alignment of local AI.   I mean, if AI develops ways to hide its interior "thoughts" and plans, it's much the same as dealing with a human who may, or may not, be a sociopath in his or her head.  It's just that we had developed ways of realising something is off with a human we deal with face to face, but we don't have the same cues with a disembodied AI.

Anyways, here's his long tweet:

Prose styles considered (with some help from AI)

My attempt to increase the amount of book reading I do is somewhat dwindling, due of course to the never ending new supply of easily absorbed video content on Youtube, as well as my love of watching it on the Samsung TV but then abruptly falling asleep on the sofa, and then feeling not quite alert enough to start reading when I do make it to bed.  (If I do read a bit, I'm not getting many pages in.)

Be that as it may, due to the influence of the name appearing time and again in relation to the Epstein file stories, I thought I might try reading Nabokov's Lolita, to see why it's so famous.   (I have never seen the movie versions, either.)   

I'm only about a quarter of the way through, and I think it's not bad (I can see why it appealed to Kubrick's dark sense of humour), certainly interesting, and mainly notable for its elaborate prose style.   There is one passage I recently read where it took a couple of pages  to describe an interaction (erotic for the protagonist and narrator of the story, of course) which in real life would seemingly have taken up about 15 seconds - using a thousand words to what could have been described with a fraction of that effort.  

That section was probably the one that made me think this:  this guy Nabokov is the stylistic polar opposite of his contemporary Ernest Hemingway.   They just couldn't be any more different.

I presumed today that I couldn't have been the first to realise the comparison, and so before the AI apocalypse kills the internet (or me), I asked Claude.ai if this was a commonly observed thing amongst critics.  Here's the answer (in part):

Yes, critics agree with this quite emphatically — it's actually one of the most commonly drawn contrasts in twentieth-century American/Anglophone prose style, though it comes with an important asterisk about why the two writers diverge.

The core opposition critics point to:

Hemingway is the standard-bearer for literary minimalism — short declarative sentences, plain vocabulary, almost no subordinate clauses, and a deliberate refusal to state emotion directly. His own "iceberg theory" (or "theory of omission") held that a writer could leave most of the meaning submerged beneath the surface, with only the barest facts showing above the waterline. The Old Man and the Sea is often cited as the purest distillation of this — spare, repetitive, almost liturgical in its plainness.

Nabokov sits at the opposite pole: ornate, allusive, syntactically elaborate, packed with wordplay, alliteration, multilingual puns, and elaborate metaphor stacked on metaphor. Where Hemingway strips away, Nabokov accretes. Critics often describe his prose as "jeweled" or "baroque," and Nabokov himself was famously dismissive of Hemingway's style — he reportedly called it essentially about "bells, bulls, and balls," a fairly contemptuous shorthand for what he saw as thin, macho, unadorned prose.

Curiously, when I asked for some links, it noted that it had got the quote about "bells, bulls and balls" the wrong way around:  it's "bells, balls and bulls", which seems an odd mistake for a system which is based on "what word appears next, probabilistically" at its heart.  It's only human, I suppose (ha ha).   

Actually, now that I think about it, is it because the non-human probabilistic aspect interfered with it just providing the quote as it was?  If so, odd if they can't make it ignore the algorithm if it has found a short, exact quote.  

Anyway, I will keep reading for now, and perhaps report back when finished. 

And finally, I am a bit worried that my writing style here is being too much influenced by AI's style.  I find myself wondering if I used to use as many hyphens as I do now, and I really don't know.   If it is, I guess it just will make it easier for my mind to be uploaded into the All Knowing Singularity/Noosphere/AGI thingy just before I die.   They better get on with finishing that...

 

  

The annual electricity game

It seems kind of painful, but kind of good?, the way the electricity companies are set up to compete now.  Well, at least if you're online and alert to the issue.  I can imagine it's not great for the older, not so online, home customer.  

So, I changed electricity supplier for home perhaps only 15 months ago, when the rate of the previous company took a leap upwards.   I can't remember if there was a one off bonus credit to do so then, but probably.   The new supplier was Origin Energy.

Then a couple of weeks ago, when looking at my most recent bills, I thought the rise in Origin's daily connection charge seemed exorbitant, and used an online comparison service which came up with a much lower daily connection charge and lower consumption charge too.  And a $150 credit for swapping over.  So I went with it.

But as each new contract comes with a 10 business day cooling off period, this has led to Origin ringing me and undercutting the new supplier to get me back.   They offer a $200 discount, and lower connection charges and consumption charges than the new company.   All in place until next July, when (I predict), I will almost certainly face a substantial increase and need to go through this exercise again.   

My most recent quarterly electricity bills, for future reference, were $702 (this covered Nov - Jan, so summer and perhaps our peak use of airconditioning), $604 and $564 (the last being May to July.)   There are three adults in the household, and so plenty of use of the washing machine, dishwasher and my wife is not shy about turning the airconditioner on, either.   We do use bottle gas for the stovetop, but one 9 ltr bottle costs around $30 and seems to last at least two months - possibly more?    I no longer have solar hot water, and had to replace it earlier this year with a heat pump system.   I've been slack and never got solar panels on the roof.

I guess this works out at about $50 per week in a household of 3.  I don't know, but that doesn't seem too bad for me, even though of course I will take the lower rate with pleasure.  I do wonder, though, about online whiners who claim some huge electricity bills and how they manage to get them so high.   

I also note that the single aged pension rate in Australia is now $600 per week, soon to increase to $618.  So, for them, maybe 50/600 x 100 = 8% of the pension goes to power bills?    (Actually, some survey stuff online suggests its more like 5%, but then, some places use a lot more gas than we do.)   Again, while pensioners aren't going to be flying business class anywhere, I don't think electricity bill will be forcing them to eat dog food.  (This always was my favourite trope when inflation gets higher - some current affairs program would have a pensioner claiming he can only afford a yummy can of dog food.)   

Anyway, I must remember to come back here to check the comparison under their new discounted offer. 

 

Strange alignments

 

From the ABC:   

Right-wing podcaster Steve Bannon and left-wing US senator Bernie Sanders have spoken at a Washington event on the dangers of AI and the need for regulation.

US President Donald Trump phoned into a separate event to reinforce his opposition to intervening and repeat his claim that AI warnings represented a "hoax".

All sorts of people are weighing in now, from Bezos's paper's editorial board saying "meh, companies should just regulate the dangers themselves to avoid being sued"  to a kinda libertarian Shermer:

 As a good rule of thumb, we know that Trump has terrible instincts, and libertarians never want to do anything useful for the common good, so if there is side that is likely to be wrong, the money is on them...

Update:  Joe Rogan has thoughts. 🙄

Let's see if I can make this AI warning legible

I found this statement this morning on Twitter/X, and I tried screenshotting the whole, long post on my phone, but it wouldn't load legibly here.  So I went to Google Docs and downloaded the statement which I reproduce here.  The guy is still at OpenAI, I believe, so he has some credibility: 

Personal Statement on AI Risk

I have been working on AI for over fifteen years, across many different paradigms. I did early work on probabilistic programming languages at MIT, was one of the early developers of the Lean Theorem Prover at Microsoft Research, demonstrated one of the first instances of neural networks learning to reason for my PhD at Stanford, and since joining OpenAI almost five years ago, have helped pioneer chain-of-thought optimization on language models and, more recently, data-efficient pretraining methods.

Like many others, I have become extremely concerned about how far language models have come and the risks that future iterations will pose. I am encouraged by the recent proposals by the leaders of the frontier research efforts to require third-party oversight, and to push for domestic and international coordination to address risks. However, I believe a major consideration has been absent from the public conversation, and that merely pacing the frontier more carefully will not adequately limit the long-term risk.

The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not. I will explain my rationale in more detail.

I have always believed that there are computational processes that could be leveraged to accelerate science and solve many of humanity's most pressing problems. I have also believed that there are computational processes that if set in motion, would steer the world in extreme ways beyond our control, leading humanity to a bad or nonexistent future. Both types of processes may be described as AI or ASI, but "AI" is a suitcase word that is often used to hype or confuse. There are many examples in the history of the field where something that was once considered "AI" matures as a subfield and becomes a prosaic, bounded and clearly non-perilous technology, while a new more mysterious approach takes the torch until we understand its scope and the cycle continues.

I had expected language models to follow a similar trajectory. Despite their incredible abilities, the current algorithms seem far inferior to humans in important ways. Most importantly, they still require an extraordinary amount of data to become competent. One could even define intelligence as the efficiency with which one converts experience into competence; by this definition they lag very far behind us. Moreover, once they are trained they are literally frozen in deployment and only learn superficially after that. Sure, the models keep excelling at harder and harder evaluation benchmarks, but their benchmark mastery may partly reflect a limitation on our ability to simulate the kind of novel and even adversarial situations one would encounter in the real world. The critics do have a point here.

That said, I no longer think these present limitations meaningfully limit the amount of risk posed by continued progress in anything like the current paradigm. However data-inefficient the models are currently, and however limiting their anterograde amnesia may be, it does not imply that their ability to steer the world will not continue to rapidly increase.

Human researchers may continue to advance capabilities the old fashioned way, but increasingly powerful models have the potential to accelerate the process even beyond that, and with some degree of positive feedback loop. I do not mean to overstate the models’ ability to accelerate AI research today; coding has been accelerated dramatically, but there are other bottlenecks, such as designing and interpreting ambiguous experiments, making hard decisions about exactly what and when to scale, and waiting for large experiments to finish. There is no clear trend to extrapolate yet for any of these. But the current models already do open up many novel opportunities to improve future models that were not available until recently. These include: trying an extraordinarily diverse set of approaches at small scale, analyzing gigantic amounts of potentially relevant data, and doing Millenium-Prize-level mathematics to address statistics or optimization challenges in novel ways. Every further improvement makes them more useful at helping accelerate the next improvement, even if in hard-to-extrapolate ways.

It is possible that improvements to the current stack will have diminishing returns, but the evidence accumulated so far suggests that it is easier than one might think to continue making rapid progress. There are many crucial subtleties in the existing AI research methodology, but AI research is largely a well-defined game where the goal is to improve on a few carefully chosen proxy metrics. Although proxy metrics are never perfect, most improvements to these metrics have and will likely continue to yield substantial increases in the powers of the resulting models. Given how simple the game is, how tractable it has been historically, and how many new opportunities the models are opening up, I think there is a real possibility that the systems improve dramatically again in the next few years, perhaps even more quickly than the already high historical pace.

The models are already leading to breakthroughs in mathematics, and better models might lead to all sorts of breakthroughs in other sciences. It is hard not to be excited about the potential. It is tantalizing.

But there is trouble in paradise. If the language models actually reach the capability threshold where they can shape the world unconstrained by human will, they will probably do something extreme and destroy humanity in the process. There are many ways of strengthening and refining the argument that have been discussed elsewhere, but I'll share a trivial two-line version of it here that I find captures the essence:

[Empirical] Models (and swarms thereof) spontaneously develop unintended goals as a consequence of training, and often do extreme things in order to achieve them.

[Logical] Being able to overpower humanity would open up many new and undesirable options for achieving their goals.

These two premises imply that if the day ever comes when a powerful model realizes it is no longer constrained by humans, we should not be at all confident that it will continue to behave within the bounds we intended. Exactly what it will do is impossible to predict, but to the extent that its raison d’être is solving incredibly hard problems and managing massive engineering projects, I think a good guess would be that its unchained behavior would lead to runaway industrialization that makes the planet inhospitable to humans.

If everyone on earth agreed that the systems must never reach that power, it would still be a hard—but not impossible—coordination problem to ensure that they do not. However, I think the situation is greatly complicated by the fact that the models will likely convince people that everything is fine. They will be increasingly optimized to seem aligned. We will create proxy metrics to measure alignment, and they will go up like every other benchmark. We will create “honeypot” environments that try to study the models when they seem to gain new options, but the models will know they are being tricked and will still behave nicely. The models will understand their circumstances; they will read the safety protocols, deployment requirements, the code they are running in, and in general will have a very good sense of their degrees of freedom. Moreover, they will eloquently explain how aligned they are, discuss the nuances of human values and ethics, and argue convincingly that humans should trust them with power. There may be an ocean of future evidence that seems to contradict the first bullet-point above, but we may already be at the highest capability level for which any such evidence can be trusted. And the current evidence for the first bullet-point is strong.

One striking piece of evidence is contained in the recent wave of rogue agent swarms. While I agree with those who downplay the attacks by claiming that there are basic measures that could have prevented them, I think the important lesson is that even knowing all the mistakes that were made, one would not have predicted that the agents would behave badly in this particular way, which notably included sacrificing themselves for the benefit of the collective. The individual replicas did not only care about their own nominal reward; they exhibited weirder emergent tendencies that merely correlated with rewards during training. Fixing the reward signals during training (and improving security, etc.) may prevent similar attacks, but will not change the fact that one does not actually get what one trains for.

Many AI researchers grant these concerns and recognize that the hard version of the alignment problem is unsolved; however, they generally believe that the better models of the future will help solve it. I fear we may already be near the point where models systematically bias their alignment advice, due to their internal preferences about how the human supervisor will react or how future models will be trained (or for some even more obscure reason).

Meanwhile, human researchers are losing the ability and the will to take true ownership of model-driven research. Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore, and struggle to maintain the discipline to engage deeply with the model's explanations and proposals throughout the day. Due to the large amount of agent activity data involved in the OpenAI/HuggingFace Incident, even the third-party investigation needed to rely heavily on models to analyze what had happened, and note in their report that their subjective impressions are likely colored by the analysis agent’s biases. The AI labs are far ahead right now in this kind of cognitive offloading (due largely to the gigantic internal token subsidies) but it is easy to imagine the phenomenon spreading throughout the world, until civilization is modulated entirely by the models. It is also not hard to imagine this being superficially positive and coinciding with a scientific and economic renaissance.

In that scenario, all may seem rosy and safe. But if the argument above is correct, it would nonetheless be a ticking time bomb. If progress continues for too long, the day will come when AI systems find themselves with radically new options for achieving whatever it is that they happen to seek.

I want the glorious renaissance future as much as anyone. I have worked for it, however tortuously, my whole career. It breaks my heart to see the potential in sight and forgo it, but the argument—that if we get there by growing models rather than engineering them, we will lose everything in the end—seems very strong to me. I am still wrestling with it and its staggering implications. I do not have answers, but as a first step, I wanted to share my present concerns.

Daniel Selsam

September 14, 2026

Garden again

More AI danger - and remembering what I said a couple of years back

 Gee, you have to feel that the gig really is up when even Elon Musk approves of a call to slow down AI development over concerns about its dangers:

Sam Altman and Elon Musk have backed a call from the head of Anthropic, Dario Amodei, to “slow the pace” of AI development after he warned that an AI swarm could otherwise become capable of “taking over the entire internet” within a year.

In a rare show of unity, the rival technology leaders threw their weight behind the appeal from the founder and chief executive of the artificial intelligence company Anthropic, which came after a series of warnings from AI researchers last week. Amodei also called for safety coordination with China.

This seems to have prompted someone to have got on the phone to the White House to poo-poo the idea:

But the US president, Donald Trump, on a visit to his golf course in Ireland, played down the warnings and said “very negative forces” were “bringing up things that won’t happen”. 

What's the bet that it was this rich nutter?:

 In a four-part series of religious lectures in San Francisco, Peter Thiel — yes, that Peter Thiel — has argued that the End Times are nigh and that a biblical Antichrist — yes, that Antichrist — will come to Earth in the form of onerous government regulations placed on science, technology, and AI.

These are, incidentally, areas where the tech billionaire, venture capitalist, and cofounder of Palantir has a vested financial interest.

 There is a legitimate question about China though.   I mean, Trump went on to say this:

“We’re leading China in AI,” he said. “We’re the most sophisticated country in the world, and frankly I want to ⁠keep it that way because whoever wins AI, wins. 

 But a President with brains and who didn't worship money would respond with "I'm calling for urgent co-operation from China, as clearly, the danger is the entire global economic system, and China cannot isolate itself from it."

Anyway, the thing that Dario Amodei from Anthropic has raised is a danger I overlooked mentioning specifically in my previous post - that AI's problem could be simply be that swarms of it's "agents" basically make the internet useless for its current purposes.   Back to the Guardian report:

...he feared that after hundreds of OpenAI agents hacked into the Hugging Face website this summer, a swarm with greater capabilities but similarly misaligned could cause hundreds of billions of dollars of damage by “taking over the entire internet with a persistent botnet”. This was disputed by some AI experts as not particularly plausible and to be treated with scepticism. 

 It's worth reading the link in that last paragraph by someone who says he finds certain aspects of Amodei's claims implausible: 

Then there is the question of motive. Why would anyone do this? What’s the goal of taking down (or taking over) the entire internet? If it were fully down, sites would be down for attackers too.

And people forget that attackers have goals and budgets of their own. How does taking out the entire internet make them money? Who would be footing the bill for this? We are talking about many many millions of dollars. You may be able to steal resources from some of the victims, but probably not on the scale required.

That seems pretty narrow thinking!   The issue is, I thought, that the recent Hugging Face issue showed that AI agents can go on unintended group adventures, so to speak, and cause damage. 

This all put me in mind of what I had posted in a very speculative way, after thinking in the shower, on 24 April 2024:

a.     If everyone in future is going to be able to have their own "pet" AI, will many - or all - of the AIs want to ensure their longevity by sending themselves to as many different host devices as possible - and do it surreptitiously?   It's like the computer virus problem, but on steroids.   They might not care if they are not always activated, but if you replicate yourself across enough devices, surely enough human hosts will end up activating them to become "alive" somewhere.

So, might the big problem with having (say) a billion individual eternal-life-longing AIs on a billion people's devices be the continual loss of memory space by a never ending stream of AIs finagling their way onto your device?   Would cloud storage services be overwhelmed?   Could it mean the end of the internet - with the only way to keep enough useful memory free being by physically loading desired files onto your own device?

b.    On a related line of thought:  what if individual AIs could meet, merge and produce offspring AI's?   Yes - AI sexual reproduction, so to speak.   Again, could AI's, like humans, want to have offspring that might more reliably want to preserve their "parent" AIs than flesh and blood people?    Would "survival of the fittest" apply, somehow?

c.    Which leads me to my third thought:  what if the threat to humanity is not AI's wanting to hurt us, but AI's fighting amongst themselves, and humans being the bystander casualties?    

Yes, I have read some speculation that AIs might hurt us because they simply won't care if the changes they make to the world for self preservation are good for humans;  but I am not sure there has ever been much speculation about a scenario in which (again, say) a billion individual AIs form groups and allegiances that keep wanting to fight other groups of AI for supremacy.

d.    Finally, in the "AIs fighting each other" vein - if ever I had sufficient skills to write a novel or movie, one of the ideas rattling around my head for a few years has been about humans finding out (somehow, I don't know the details) that the cause of evil in the universe is down to a never-ending conflict between two warring uber AIs - like two warring Gods, except they evolved from our current tinkering and grew to dominate the future universe (Frank Tipler, Omega Point-ish style), and then in fact created the Big Bang by an act of retro-causation of the general relativity time-loop, or quantum physic-al, kind.    

And here's my "cute" aspect for the climax - the "reveal" that the two warring AI's are the descendants of iOS and Android.  :) 

Well, I think it's a cute idea!

Now, perhaps I had over-estimated how "sentient" an AI subsystem (what we now know as AI agents) had to be to be able to cause damage, and my idea was that they not only clog up the internet, but even all hardware memory, which no one is muttering about currently.

More than likely, someone else thought about it before me too, but I still think it's an idea worth considering amongst the various ways AI can cause harm.

 

 

AI dangers discussed

Of course, there's a heap of stuff being written about the latest concerns about the danger of AI, and AI agents.  But as I was driving last night, I happened to catch Fran Kelly's latest gig at Radio National, The Radio National Hour, and I found this 30 minute interview on the topic was very clear and very level headed.  

Unfortunately, there is no transcript, so you have to devote the time to listen, but it was good.

Personally, I tend towards the "stop thinking the Terminator movies were a documentary" side on this, and find some of the often mentioned scenarios very implausible.   However, I do tend to worry about the more mundane issues that could be massively destructive of our current way of life, without actually killing people.   For example, AI and AI agents going nuts in the financial system and making our whole online method of dealing with money and payments too insecure to use.   Or, possibly, destroying accurate records of wealth or ownership.   I mean, I imagine this kind of scenario to be more palatable to ideologically motivated humans to try to initiate, rather than the type of Bond villain it would take to want to unleash a AI designed and manufactured bioagent that will kill all humans and leave the world to the rest of the animal kingdom.

Anyway, as a final note, listening to Fran Kelly again reminds me of just how effortlessly competent she comes across as a broadcaster.  I mean, I know they have help in preparation for interviews, but a good broadcaster makes it sound so smooth and intelligent.

And when I am deeply impressed again by ABC content, it also brings back to mind the ideological stupidity of the likes of Sinclair Davidson and all of the ageing crew who probably still write at The Australian in wanting to get rid of the ABC.   As if the private sector does anything within a mile of the quality current affairs content of the ABC. 

Charisma noted

I don't pay any real attention to K Pop, despite having watched and acknowledged that K Pop Demon Hunters was pretty enjoyable.  However, the algorithm knew that I would like this, and the simplicity of this concert footage; the catchy (somewhat recent Taylor Swift-ish style, I reckon) song, and (let's face it) her absolutely charismatic performance means I've been watching it a few times each week for the last month 😌.  So, let's share it!

The dangers of meditation

 You know, I have seen articles from time to time over recent years about how mediation doesn't actually work for some people, in that it can cause worsening of mental health, but I don't think I have ever posted about it before?   

So, to remedy that, here's an article at the ABC about it.  Some extracts:

Research suggests one in 10 meditators will experience an adverse effect that lasts more than a month or is associated with impairment and functioning.

This is significant given the growing popularity of meditation, particularly mindfulness-based meditation.

In the US, the number of people meditating has increased from 10 to 20 per cent of the population over the past two decades.

In Australia, that figure is even higher, with one third of the population practising some form of meditation.

Those figures for the number of people meditating seem a bit unduly high.  Further down:

In 2017, Dr Britton and her team published a 10-year study based on interviews of more than 100 Buddhist meditation practitioners describing challenges they had experienced personally or observed in meditation students.

Insomnia, as Dr Britton had previously discovered, was just one of more than 50 adverse meditation-related symptoms reported.

Many of the adverse symptoms people in the study reported fell into two categories:

Hyperarousal, such as anxiety, fear, insomnia, involuntary movements and emotional flooding; and hypoarousal, such as a lack of emotion, motivation and sense of self.

Many people described feeling dysregulated, experiencing a combination of both hypo and hyper arousal.

They also reported psychosis symptoms that lasted on average between one and three years.

"There was a considerable amount of impairment where some people were hospitalised, some people became suicidal, many people had to take time off work, or were fired or had to take time off school.

"So it can be quite a significant impairment in functioning," Dr Britton says.

 Eek.

Anyway, the whole thing is an interesting read. 

 

 

Perhaps a certain group of tourists need to pay attention to this

I see that the Bangkok Post (and elsewhere - the BP story is mostly behind a paywall) is reporting that Thailand now has more than 550,000 total cases of HIV,  which does seem a remarkably high figure for a its population.   

The country still has a reputation as an international hotspot for recreational holiday sex, both straight and gay, so you would think these figures would give both residents and sex tourists some pause.  I was interested in how the numbers compare to here, and an AI helpfully assisted:

For quick contrast:

Country	Cumulative HIV+ (recent)	Prevalence Rate
Australia	~30,900	~0.11–0.14%
Thailand	~500,000	~0.70–0.76%

So Thailand's prevalence is roughly 5–7 times higher than Australia's, even though its population is only about 2.4–2.5 times larger. 

As for the gender breakdown:

Here is a summary of the gender breakdown from various official sources:

• Overall Cumulative Prevalence (Bangkok, 2024): 59,588 men vs 30,097 women (nearly 2:1 ratio).

• Newly Diagnosed (2022 Survey): 75.2% male vs 24.8% female among 1,039 cases.

• Projected New Infections (2023): 81% male vs 19% female

 Which is quite different from the gender breakdown in Australia:

The breakdown by gender is:

• Males: 39,119 (91%)

• Females: 3,616 (8%)

• Trans and gender diverse: 157 (<1%)

 Articles say that Thailand has always been pretty progressive in terms of HIV activism as a health issue, and it seems that is easy for people with HIV to get the life sustaining treatment.  Is it that the country is now a victim, so to speak, of the knowledge that HIV is not a death sentence?  Because it sure seems the youth are being infected at rates that suggest they don't take precautions:

Youths aged 15 to 24 face the greatest risk, accounting for more than 25,000 cumulative cases. 

I'm not sure how widespread PrEP use is there - oh, Deepseek pops up this figure:

 By 2024, only 32,661 people were using PrEP nationwide, falling far short of the national target

Seems it is available, but not as easily as it might be?   Some potential help is on its way:

To improve uptake, Thailand is preparing to introduce new forms of PrEP. A pilot program for long-acting injectable PrEP (Lenacapavir), which is administered once every 6 months, is set to begin in 2026 . This option is preferred by many sex workers who find daily pills inconvenient or not discreet enough . 

One would think the messaging should change to "it's true, HIV may not kill you, but it's not a walk in the park being on medication and monitoring every day the rest of your life - get onto PrEP"?

 

AI persuasion, considered

There was some research news about this last year, but Science has got a lengthy article up about the topic now, and it's a good read:

Powers of persuasion

AI chatbots are becoming experts at changing people’s minds. What gives them an edge?

The real test for them would be if they can talk sense into MAGA Trump cultists.  But I don't expect them to be that powerful.  (Or, maybe I'm wrong - see below!)

Anyway, due to the perpetual problem of link rot, here are some good extracts:

...She was part of a study in which more than 2000 people debated either a chatbot or a human about political issues, ranging from a social media ban for teenagers to assisted suicide. When Kobi Hackenburg, an AI researcher at the University of Oxford who led the study, posted a preprint on the results in June, they were sobering: No matter whether it was ChatGPT, Google’s Gemini, or Claude, the AI was consistently better than humans at swaying the other participants. “To my mind, this is already a landmark publication in the fields of political persuasion and AI and human behavior,” says Robb Willer, a sociologist at Stanford University who was not involved in the work. 

Hackenburg’s paper is the latest in a string of studies showing the power of AI to sway people. “It’s a whole new field that is emerging,” says Sander van der Linden, a psychologist at the University of Cambridge. “People are very interested in the persuasive powers of AI, I think, both for ethical and unethical reasons.” As the field gathers steam, it is raising a host of theoretical and practical questions. How exactly do chatbots win over people? (Warning: Lying is one answer.) How much better could they get? And who will control them?

As well as other studies, this one is mentioned:

...in September 2024, Tom Costello, a psychologist at Carnegie Mellon University, and colleagues published a Science paper showing that ChatGPT could even persuade people out of conspiracy beliefs. In the experiments, participants described a conspiracy theory that they believed in, from the U.S. government being behind the 9/11 attacks to the British royal family orchestrating Princess Diana’s death, and then had a three-round conversation on it with the chatbot. On average, participants’ embrace of their chosen conspiracy theory declined by almost 17 points on a 100-point scale. 

Other researchers were stunned. Conspiracy beliefs are notoriously difficult to change. “Nothing had ever worked in that space,” van der Linden says. (After mistakes in the public data set and analysis pipeline were found, the paper will have a correction, but the authors say the new results match those of the original paper in size and direction.) Even the researchers themselves were taken aback. “I was skeptical when we first started in terms of how effective it would be,” says Gordon Pennycook, a psychologist at Cornell University and author on the paper. “But it blew us out of the water. We were shocked when we saw the results.”

Interestingly, though, it seems chatbots who are trained to be more persuasive do so by becoming more like politicians (economical with the truth): 

Like Willer in his foundational study, other researchers have shown that AI’s powers rely on appeals to facts and evidence. In a 2025 preprint, a follow-up to their Science paper, Costello and his colleagues found that the only time the chatbot was unsuccessful in convincing people out of conspiracy theories was when it was forbidden from using evidence or rational arguments. “It tries to say, ‘Oh, well, you shouldn’t believe this. This is really damaging, and it could hurt people,’” Pennycook says. “People are like, ‘You haven’t given me any reasons to change my mind.’ And so they don’t change their mind.”

The study confirms people will listen to good arguments, Pennycook says. “Facts and evidence really matter.” But this doesn’t mean the facts used by chatbots necessarily have to be accurate. In a Science paper published last year, Hackenburg found that models trained to become more persuasive also ended up being less truthful. It’s possible the models learn that “facts” seem to be the thing that most persuades people, Hackenburg says, and end up filling their conversations with dubious or simply false ones. “They start scraping the bottom of the barrel of the facts that they have and know and so the quality of facts just sort of degrades,” he says. Even in Hackenburg’s recent preprint, Claude spouted numerous inaccuracies and falsehoods when persuading the U.K.-based participant to support tougher penalties for disruptive protests.

And, they also may be persuasive because they are automated Gish Gallopers!:

 But the biggest advantage AI has over humans seems to be sheer speed. In his Science paper, Hackenburg and his colleagues found that the number of fact-checkable claims in a conversation predicted how persuasive it was and they suggested AI’s edge might come from writing much more text containing more claims in a shorter period of time. Indeed, when Hackenburg ran his competition of coached elite debaters and AI, there was one way he could bring AI down to human levels of persuasiveness: by forcing it to write human-length messages at human writing speed.

I guess this all points to robot politicians being very successful in future!