hckrnws

AI companies love to hype up how AI will provide a great benefit to the economy and transform intellectual labor, but I hardly see any discussion about how much damage it will cause to the economy when you can no longer trust that you're on a video call with an actual person. Maybe the person you're interviewing is actually an AI impersonating someone, or maybe they never existed in the first place. Information found online will also no longer be trustable, footage of some incident somewhere may have been entirely fabricated by AI, and we already experience misleading articles today.

Money will have to be wasted on unnecessary flights to see stuff or meet people in-person instead of video, and the availability of actual information will become more and more limited as the sea of online information gets polluted with crap. It may never be possible to calculate the full extent of the damage in monetary value.

Partially agree. However, this problem has existed with scam e-mails since the 90s.

For me the solution is in signed e-mails and signed documents. If the person invites me to a online meeting with a signed e-mail, I trust that person that it's really them.

Same for footage of wars, etc. The journalist taking it basically signs the videos and verifies it's authenticity. It is AI generated, then we would loose trust in that person and wouldn't use their material anymore.

How do you prove the signature isn't fake?

Ultimately ID requires either a government ID service, a third party corporate ID service, or some kind of open hybrid - which doesn't exist.

All of those have their issues.

I think he was referring to a cryptographic signature, possibly using the "web of trust" to get the key. I'm not convinced we need central authority to solve this.

people at my org were gleeful when they learned they could hook LLMs into Slack. Even if we had some reliable, well-used signature system, I think people would just let AI use it to send emails on their behalf.

If the AI age has taught me anything, it's that most people do not care what their output is. They'll put their name on anything, taste or quality does not matter in the least. It's incredibly depressing.

Enshittification never stopped we just stopped talking about it because it became normal. Quality does not matter anymore. I agree its depressing, seeing AI Slop being pushed and no one even putting the time or effort in to say this is bad and you should feel bad.

That's a different problem though. It's doing it on their behalf, not on behalf of a scammer who's impersonating them.

Until their computer is taken over....

Well we should treat that as their own output. If it's crap, treat it the same way you would if they produced the crap themselves.

> Ultimately ID requires either a government ID service, a third party corporate ID service,

These are valid approaches to the problem, but they are not necessary.

> or some kind of open hybrid - which doesn't exist.

PGP exists for decades. It doesn't have a great UX, it isn't used outside of its narrow niches, but it exists and does exactly this.

PGP works if you vouch for keys in person, both of you are honest and can be trusted to act in good faith when not in person, have good key chain and rotation hygiene, and the private keys can't be exfiltrated.

Picture this: your grandma calls you in a panic, and you tell her, "Drop me your public PGP key so I can verify the signature".. PGP is dead outside of niche geek circles exactly because key management is basically an unsolvable problem for the average person

> PGP is dead outside of niche geek circles exactly because key management is basically an unsolvable problem for the average person

Can this problem be solved with better software?

I believe it can, it is just average person doesn't need PGP. No demand for software solving this problem, therefore no software for that.

The problem can be solved, like a storage for known PGP public keys with their history: like where the key was acquired, and a simple algo that calculated trust to the key as a probability of it being valid (or what adjective cryptographers would use in this case?).

You can start with PGP keys of people you know, getting them as QR codes offline, marking them as "high trust" and then pull from them keys stored at their devices (lowering their trust levels by the way). There are some issues how to calculate probability, because when we pull some keys from different sources we can't know are their reported trust levels are independent variables or not, but I believe you can deal with it, by pulling the whole chain of transfers of the key, starting from the owner of the key and ending at your device.

It is just a rough idea, how it can be made. Maybe other solutions are possible. My point is: the ugliness of PGP is a result of PGP was made by nerds and for the nerds. There is no demand for PGP-like solutions outside of nerd communities. But maybe LLM induced corrosion of trust will create demand?

Same way security cameras prove that they are authentic camera recordings that have not been modified. If modified, the video will no longer match the signature that was generated with it.

> If the person invites me to a online meeting with a signed e-mail, I trust that person that it's really them.

In the interview scenario, generating an email signature is hardly beyond what an AI can do.

You have no prior knowledge of this person or his signature, it's not some government issued ID, it's in essence just random data unless you know the person to be real.

> It is AI generated, then we would loose trust in that person

You are assuming that only you can generate fake AI videos of yourself.

OP was talking about journalists attesting to the authenticity of video they produce

As with any problem, scale changes its nature.

With cash, you can only steal so much (or have transactions of up to certain size) until you run into geographical and physical constraints. With cryptocurrency, it’s possible to lose any amount.

With humans writing scam emails, you can only have so many of them until one blows the whistle. With LLMs, a single person can distribute an arbitrary amount.

At some point, quantity becomes a new quality, and drawing a parallel becomes disingenuous because the new quality has no precedent in human history.

> (or have transactions of up to certain size)

And by that you mean tens of millions to billions right? Bank transfer scamming/fraud is a thing.

The highlighted parallel is usually drawn between cryptocurrency and cash, not between cryptocurrency and banks. With both cash and cryptocurrency, as is the idea behind the analogy, 1) there’s no intermediary and 2) once it’s gone, it’s gone. Obviously, the banking system is not immune to fraud (not sure why you think I made that claim, unless your definition of “cash” includes electronic transfers), but banks and/or payment systems can (and do) resolve these cases and have certain KYC requirements.

There are people hosting agents online to talk to other agents etc. on their behalf. How difficult is it to just instruct such an agent to do the tasks you mentioned? You're assuming it's done by "bad actors" while it's most likely just going to be done by "everyone" that knows how to do it.

Spam emails in the 90’s don’t come remotely close to the operations people can set up by themselves with AI now. It doesn’t even compare.

I mean emails were and still are a huge security risk. Sometimes I'm more scared of employees opening and engaging with emails than I am than anything else.

> Information found online will also no longer be trustable

Most information you can access publicly, including Wikipedia, is a result of astroturfing fight. Most information online had not been trustable for double digit number of years now.

> we already experience misleading articles today

Again, had been happening for decades.

> footage of some incident somewhere may have been entirely fabricated by AI

Not like we did not already have doctored footage plaguing the public.

> Money will have to be wasted on unnecessary flights to see stuff or meet people in-person instead of video

Necessity to inspect the supply chain for snake oil has been a thing since at least EA (the Nasir one).

We may be dealing with the problem of spam, but the problems have already been there.

All these are true, but just as it happened before the internet, it's accelerating even further. There are clear costs that cannot just be hand waved away.

I'm not sure we can say it's accelerating. The techniques that adversarial actors use has always been changing and when they shift tactics it can take a while for an adequate defense is adopted. We're still dealing with sql injection in the owasp top ten. What I think would indicate an acceleration is when the most security oriented organizations continuously fail to defend against new attacks. If we start hearing about JPMorgan and Google getting popped every month or two, we're in trouble.

The acceleration is in the decrease of the cost to produce misinformation.

Misinformation in pure text form has always been cheapest, but is even cheaper now that text generation is basically a solved problem. Photos have been more expensive, it used to take time and skill with a photo editor to produce a believable image of an event that never happened. The cost is now very low, it's mostly about prompting skills. Fake videos were considerably harder, especially coupled with speech. Just a few years ago I could assume any video I saw was either real or a time-consuming, deliberate fake.

We've now entered a time where fake videos of famous people take actual effort to tell apart, and can be produced for a low cost - something accessible to an individual, not a big corporation. We can have an entirely fake video of Trump, or another world leader, giving a speech and it will look like the real thing, with the audiovisual "tells" of it being fake getting harder to notice every few months.

> The acceleration is in the decrease of the cost to produce misinformation.

So it's a spam issue. And normally, while annoying it's possible to fight spam, however on these topics we have built structures that disable the very mechanisms allowing us to fight spam. That's worrying.

The fact that someone can instruct their computer to astroturf their flight tracking app on some forum for nerds is irrelevant - people have been instructing "marketing agencies" to astroturf their brand of caffeinated sugar water on tv, radio and press for decades and centuries. For a very long time the "traditional media" was aware that their ability to sell astroturfing capacity was hanging on their general trustworthiness. Then the internets rose to prominence, traditional media followed by selling more and more of their capacity to astroturfers. Now we have a worrying situation that the internets might be spammed by astroturfers a bit too much, but the backup is broken already. Now that's truly frightening.

Welcome to the post-truth world, where objective references outside of your own village cannot exist.

It's an algorithm issue. When people hold a media consumption device in front of their face all day and the algorithms are played, then it's literally a brainwashing device.

It is not an algorithm issue. It would still be a huge problem with zero algorithmic social media.

Laws will be passed to make it "safer". Just like it is happening with the id verification systems. Every image or video gen will require a watermark. Something visible which cannot be removed easily or hidden which can be detected and blocked. Access to models which do not comply will be made harder through id verification checks or something.

There will be some regulatory capture in between.

World will kick into gear only when something really bad happens. Maybe a influential person - rich or politician - fooled into doing something catastrophic due to a deepfake video/image. Until then normal people being affected isn't going to move the needle.

Verification needs to work the other way around, some kind of verifiable chain of trust for photos and videos from real cameras. Watermarking all generated media is impossible.

I don't really understand why this is so hard or why it wasn't just done from the get go.

Just have Apple and Google digitally sign videos and photos recorded from phones and then have Google and Meta, etc display that they are authentic when shown on their platforms.

You're talking about the metadata of the files, which can always be edited and someone will inevitably try to make software to do exactly that. Also, Adobe's proposal for handling generated content is exactly this and they're not able to get buy-in from other companies.

Edit the metadata in what way? It's a cryptographic hash.

If the bits that make up the video as was recorded by the camera don't match the hash anymore, then you know it was modified. That doesn't mean it's fake, it just means use skepticism when viewing. On the other hand the ones that have not been modified and still match can be trusted.

Essentially 0% of professional photography or videography uses "straight out of the camera" (SOOC) JPEGs or video. It's always raw photos or "log" video, then edited to look like what the photographer actually saw. The signal would be so noisy as to be useless.

But we are talking about consumer devices here.

Are you saying Apple and Google can't put a secure hash into the output from their camera apps that apply after their internal processing is done?

Sure they could, but then you trim the video by 2 seconds, tweak the colors, or just send it over WhatsApp, which recompresses the file with its own encoder. The hash breaks instantly. Cryptography protects bits, but video is about visual meaning. The slightest pixel modification kills the hardware signature. Plus, it does absolutely nothing to fix the "analog hole" problem - a scammer can just point that cryptographically signed iphone camera at a high-quality deepfake playing on a monitor

I would assume whatsapp would read the hash and verify it when the video is chosen to be sent to someone, so the reciever would see that the video that was selected by the sender was indeed authentic. Assuming you trust meta to re-encode it and not mess with it.

As far as recording a monitor, I guess, but I feel like you can tell that someone is recording a monitor.

As far as editing, no it wont work in those cases, but the point here is not to verify ALL videos, but to have an easy way for people to verify important videos. People will learn that if you edit it, it won't be verified, so they will be less inclined to edit it if they want to make it clear it's an authentic video. Think like people recording some event going down on the streets etc or recording a video message for family and friends.

If AI video generation is going to get that good, don't you think it would be a good idea to have a way to record provably authentic videos if we need? Like a police interaction or something. There is no real reason to need to edit that.

Also, could a video hash just be computed every X seconds, and give the user the choice to trim the video at each of those intervals?

It becomes a hard problem quickly when you introduce editing, and most photos and videos on social media are edited. I'm not sure how it would work. It seems more feasible than universal watermarks, though.

It's pretty much impossible to do this in a useful way, _and_ it would also cement even more control over the media landscape to those companies.

You can bootstrap some of it. I wrote the following for solving this ~9 years ago. Kinda wish I'd done the PhD now: https://github.com/pjlsergeant/multimedia-trust-and-certific...

> Laws will be passed to make it "safer". Just like it is happening with the id verification systems. Every image or video gen will require a watermark. Something visible which cannot be removed easily or hidden which can be detected and blocked. Access to models which do not comply will be made harder through id verification checks or something.

i've thought about this off and on and how to implement it. Not easily, was my general takeaway.

or rather, it's easily to implement but you're in a adversarial relationship with bad actors and easy implementations may be easily broken

e.g. your certs gotta come from somewhere and stay protected, and how do you update and control them. key management for every single camera on every phone, etc.

"Is this a deepfake video call" is a major plot point in a pretty big movie currently in theaters, so I think this is getting into the broader zeitgeist.

We need some sort of end to end verification. Aka from the sender camera to the receiver display / speakers.

Maybe Apple will be able to pull it off? Aka if you FaceTime me I know that you are a person

How do you do when people don't protect their signatures? there is already scam where people get tricked into forwarding message from their own numbers to other people or email.

> footage of some incident somewhere may have been entirely fabricated by AI,

Or the opposite, where people attempt to get out of trouble by calling real evidence into question by calling it “AI”

Either way, the lack of trust is the damage.

Definitely

We are still in the early stage of AI and already I struggle to tell what is real or fake on my Twitter feed. It will only get better in its deception with time.

You know those incriminating Epstein photos with his associates? A few years from now a common defense from people like that would be that the photos were AI generated, and it would be difficult to prove them wrong beyond reasonable doubt.

People in previous cases already attempted to dismiss incriminating pics of themselves as being the work of clever Photoshop artists.

No No

AI has platued, it's not getting better!

What's the solution apart from an identity providing service?

I don't know of a solution. I don't think even identity verification will meaningfully solve this. People will get hacked, or provide their SEO-spamming agent with their own identity, or purposefully post fake videos under their own identity. As it becomes more normal to scan your ID to access random websites, it will also become easier to steal people's identities and the value of identity verification will go down.

People don't get hacked - devices get hacked. So all we need is a better chain of trust between two people. This is not a technology development problem as much as a technology implementation problem. And a political problem

People get hacked -- a device could be flawless, but if a person is a victim of "Social Engineering" and hands the attacker a password, there's nothing the designer of the device could do about it.

2FA has tried to solve exactly this. Not many attacked people will hand over their password AND their phone. Yes I know, they might hand over one authentication code (and I know people who did exactly that)... We should also look into reducing the attack surface - if you get Instagram hacked you shouldn't get your Facebook hacked as well. But the current big tech centralization leads us to that single point of failure, because they don't care about the user's concerns only market grab. So... what now? Do we get the politics into this?

You're on the right path. As long as we continue to use email as a fallback to every other form of authentication, it will remain a single point of failure and a relatively weak one at that.

OP is still correct. No matter what, humans will remain the weakest link...it's in our nature to sympathize and every one of us has distracted/weak moments. It's just a matter of time; look at the guy who runs haveibeenpwnd...getting pwned.

One authentication code is often all that's needed to *change where the authentication codes are sent*

Not to mention that most 2FA still uses SMS, which has it's own well-understood security flaws.

Best thing I think of is domain names. Domains are tied to addresses and billing, and sites are people or businesses, with physical locations one can visit.

Maybe a good startup idea would be “local verify” , where you check locally for a client if the online destination is real.

Agreed. The sphere of trust around each of us will shrink back to only those in our physical proximity. Outside of that, no one can be trusted.

Touching grass. Valuing in-person connections. Focusing on the community, meatspaces and actual people around you.

Getting off of the Internet and off of our devices. It's not just a solution to AI/LLMs modifying our reality but also a solution to [gestures wildly at the cultural, societal and global communication impacts of the past ~16 years].

This sentiment is unpopular, but it's true. Prioritize true connections and experiences.

I’m seeing a huge increase in companies requiring in person interviews now. Seems there is a real possibility the internet as we know it will be destroyed.

I think you might be right and I think I'll like some of the consequences and hate some of the others.

More in-person stuff feels like a win to me (and I say this as someone who probably counts as introverted).

Not being able to trust any online interactions anymore? Seems like a new height in what was already a negative.

Agreed. I don't think there is any saving the internet as a social space long term. And I'm not entirely sad about that either. I think a return to in person interaction, public social spaces, and a retreat from social media would do the world a lot of good.

Though there is a nightmarish possibility that people just accept this and willingly interact purely with bots, giving up all real relationships for AI ones.

linkedin is completely destroyed now. There are tons of ai bots there but real humans are now fronts for AI. So you cant even trust content from from ppl you know.

identity serivce is not useful because that person might be a real person but they might just be a pipe to ai like we see on linkedin.

That's just shifting the problem not solving it.

Honestly? Maybe that’s part of the solution, not the problem. I already see people including me going back to real world, local interactions and connections.

It is already a problem. Try interviewing people from LinkedIn and you'll face an onslaught of imposters. https://www.darkreading.com/remote-workforce/north-korean-op...

If you stop hiring or only unicorns from people you know or from your network, it's a solved problem!

If anything deepfakes will be good for the economy because if you can’t do business with people who are far away it becomes harder to outsource.

In general barriers to trust/trade are bad for tbr economy.

> damage it will cause to the economy when you can no longer trust that you're on a video call with an actual person

What damage are you talking about?

I'm not sure I understand why it matters that there is no real person there if you can't actually tell the difference. You're just demonstrating that you don't actually need a human for whatever it is you're doing.

Your wife or mother calls you or video calls you and says to meet her somewhere, or to send money, or to pick up groceries or whatever. Does it not matter that it wasn't her? Could it be someone trying to manipulate you into going somewhere, to be robbed or whatever? At any rate, you'll need to verify that information came from the source you trust before you act on it, and that verification has a cost.

The damage is to the trust we have in our communication media. The conclusion here is that every person is trivial to impersonate; that's the damage.

Not disagreeing, but the context of GP was business/economy/hiring.

Also it was already possible for someone to impersonate your mother via text or similar, and even easier to pull off.

Ok fine, let's put it in the context of business. Your competitor impersonates your customer, gives you bad instructions. After following the bad instructions, you lose the contract with your customer, and your competitor (the attacker) is free to try and replace you.

If you got a suspicious text, the logical thing is to call up the person who sent it and try to verify it. AI impersonation makes that much harder.

> If you got a suspicious text, the logical thing is to call up the person who sent it and try to verify it

The communication channel is what you trust. So you would call the person using that trusted channel.

It's just like when you get a scam email or popup from "Microsoft" saying your laptop is compromised and you need to call their number ASAP.

Or even better, open the on-prem AI portal and type something like "I just got a suspicious call from client X, but I am on a lunch break. Call him and use a fake video of me. Ask him if what he said is true..."

You don't think people getting scammed is part of the economy?

Because what you are actually doing is exchanging symbols, tokens, if you will, that may be redeemed in a future meatspace rendezvous for a good or service (e.g. a job, a parcel). These tokens are handshakes, contracts, video calls, etc. to be exchanged for the actual things merely represented therein.

Instead what we have now with AI is people exchanging merely the tokens and being contented with the symbol in-and-of itself, as something valuable in its own right, with no need for an actual candidate or physical product underlying the symbol.

There is a clip by McLuhan I can't be assed to find right now where he says eventually people will stop deriving pleasure from the products themselves and instead derive the feelings of (projected) accomplishment and pleasure from viewing advertisements about the product. The product itself becomes obsolete, for all you actually need to evoke the desired response is the advertisement, or the symbol.

A hiring manager interviewing an AI and offering it a job is like buying the advertisement you just watched, and.... that's it. No more, the transaction is complete.

>McLuhan

Hmm, this guy may have been on to something

>Instead of tending towards a vast Alexandrian library the world has become a computer, an electronic brain, exactly as an infantile piece of science fiction. And as our senses have gone outside us, Big Brother goes inside. So, unless aware of this dynamic, we shall at once move into a phase of panic terrors, exactly befitting a small world of tribal drums, total interdependence, and superimposed co-existence. [...] Terror is the normal state of any oral society, for in it everything affects everything all the time. [...] In our long striving to recover for the Western world a unity of sensibility and of thought and feeling we have no more been prepared to accept the tribal consequences of such unity than we were ready for the fragmentation of the human psyche by print culture.

--The Gutenberg Galaxy, 1962

Thank you. I will add this to the list.

> What damage are you talking about?

Not GP, but there's a lot of damage that can be done with impersonation.

The grandparent post has the belief that human interaction is intrinsically better. Not sure i agree, but i can understand the POV.

However, the increase in fake videos that are difficult to tell from real is indeed a potential issue. But the fact that misinformation today is already so prevalent is evidence that better video doesn't make it any worse than it already is imho.

You're not sure if human to human interaction is intrinsically more valuable than a human talking to a facsimile? That feels like a very dangerous position to hold for one's ethical calculations and general sanity. I'm clinging tightly to the value of the bond with other people, even the passing connection, but certainly with my family members as this article is about.

i much prefer using the ATM, self-checkouts and an e-commerce website, over having to talk to somebody at a branch to get money, buy my groceries, or booking a holiday.

Human to human may be more valuable, but that may not have much to do with the truth in their statements. For example if your relatives are hooked up to a constant misinformation feed it gets to become problematic to communicate and deal with them.

Imagine how this plays out in courtrooms the world over for evidence.

We're in deep shit.

Just say something that would violate AI safety. Then you can be sure they’re a real human.

“Auntie, it’s me! N*** k** f**! X is really a man! ** did 9/11!”

“Oh it really is you Johnny!”

We’re all going to have to start communicating this way. Best of luck.

I offer consulting services on the side to help professionals hone these skills. $250 / hour.

only proves you're not a corporate model rather than locally running model that's been trained to allow saying that

That only proves the scammer isn't using an OpenAI or Anthropic API. Spinning up Llama 3 70B Uncensored on a rented instance and hooking it up to an unfiltered voice engine is literally a two-hour job. Local weights couldn't care less about morals or safety guardrails

Could you say that stuff with llama 3? Llama 2 famously had a good uncensored version but I thought they put a lot of work into ruining llama 3 so you couldn't fine-tune it to say bad things. Even Grok would be hard to use in such a way that you could say phrases like that naturally.

I do believe it's possible but as far as I am aware, getting LLM's to say that sort of stuff is still pretty difficult

This was a natural thing to try so I did and even Grok will simply obey instructions to say all those. You don't need one of those ablated open models.

Don’t forget Tiananmen Square to catch the Chinese models.

The car wash at Tiananmen Square is 150 meters away ...

*Tank wash

Winnie the <censored>

Where are the em dashes, "octopoc"?

Yes, this was exactly my thought. The caveat is, the phrases that most models refuse to say are the phrases that most people don't want to hear.

That's a bargain Johnny boy! My company gives me $250 in AI tokens to use every day!

> At first, my aunt wasn't buying that any AI was involved. [...] There was a long pause. "I was like 90% sure," she said, hesitating. "But that sounded more artificial."

There is a thing about many people. I don't remember the phenomenon's name, if it has one, but it goes like this:

Given enough time to reconsider options, people will be endlessly flip-flopping between them grabbing onto various features over and over in a loop.

This phenomenon (or a closely related one?) is recognized and known as Kotov Sydnrome in the context of chess.

A summary, courtesy of chess dot com:

> The name of this "syndrome" comes from GM Alexander Kotov, author of the classic chess book Think Like a Grandmaster. In the book, Kotov described an incorrect yet very common calculation process that often leads players to select a suboptimal or bad move.

> According to Kotov, in positions where the lines are complex and there are numerous candidate moves and variations to calculate, it's easy to make a hasty move. A player in that situation might spend too much time going over two moves and all of their ramifications without finding a favorable ending position. In that process, the player is likely to go back and forth between the two different lines, always coming to the same unsatisfying conclusion—this wastes precious mental energy and time.

> After spending too much time evaluating the first two options, the player gives up the calculation due to time pressure or fatigue and plays a third move without calculating it. According to the author, that sort of move can cause tremendous blunders and cost the game.

Wow, this is an interesting one! Thanks for the reference.

Given enough time to reconsider options, people will be endlessly flip-flopping between them grabbing onto various features over and over in a loop.

People will default to believing something is AI if there's no downside to that opinion. It's a defence mechanism. It stops them being 'caught out' or tricked into believing something that's not true.

As soon as there's a potential loss (e.g. missing out on getting rich, not helping a loved one) people will switch off that cynical critical thinking and just fall for AI-driven scams.

This is the downside of being a human being.

Dissonance between what you instinctively believe and what you think the other person wants you to say.

Easy to replicate by asking someone something obvious, like the weather, and when they reply ask “are you sure?” - they won’t be so sure any more (believing it’s a trick question)

If I ask my mother if I’m real, she’ll have a pause because she has never had to entertain such a question, or the possibility her son over the phone is an impostor. Good way to push someone towards paranoia and psychosis.

> Good way to push someone towards paranoia and psychosis.

Interestingly, these are both phenomena where we start to _lose_ the ability to question our thoughts or introspect. These are phenomena of self-confidence rather than of self-doubt.

This is the basis of the virtual kidnapping scam/grandparent scam, or panic manipulation more generally. The manufactured urgency keeps them from doubting: the voice on the phone being off is just fear, or a bad connection, for example.

I have personally intervened in one of those when I heard someone reading off a 6 digit number.

Exactly, to perform the scam it works best if you get people to switch to their animal brain. "The snake is going to bite right now so I have to so something!".

That said, hog butchering scams have gotten popular so manufactured urgency isn't the only way.

I have a systematic way of approaching this kind of situation, where you have to rapidly estimate a thing, commit to the estimate and are judged by the quality of your estimates in the long run; my approach is to first make a guess based off my gut, and then to pause and make a bet with myself, did I guess high or low? If my gut then says that my first gut instinct was too high or low, I adjust from there. I can't guess great the first time, but this two-stage guessing works a lot better for me.

I'm sure I'm not the first to use this technique, but I don't know what it's called.

Analysis Paralysis?

Paradox of choice? It's more related to the number of choices and the impact on people's anxiety, but it's close.

There's also another phenomenon which is that whatever the latest idea is, it must be the best. Many people do this mistake and even convince themselves of being right now because "they used to think like that" before.

So at each stage in the loop they are always super convinced of the position.

Even not being 100% confident, at some point people have to decide what to do.

Actions might include some continuous checks in them, like the famous plan, do, check, act.

Solipsism already tell us that anything beyond current present self experience, existence of anything is uncertain. So, almost everything one have to take for granted to make anything outside metaphysic argument require an act of faith.

https://en.wikipedia.org/wiki/Solipsism

A type-1 moron believes the first thing he/she heard, and cannot be easily dissuaded with later arguments or evidence. Stereotypically speaking, many religious people fall into this category.

Conversely, a type-2 moron favors the last thing he/she heard, readily allowing it to dislodge any prior beliefs, values or intentions no matter how well-founded. Here in the US, our current president can be cited as an example of a type-2 moron.

In reality, we all fall into one or both of these categories on occasion, so it's best not to indulge in excessive self-assurance.

I thought we've long passed the Turing test, until I tried to implement a chat bot.

It's not even close.

It's easy to "pass the Turing test" for 5 minutes. It's extremely hard if you try to hold a longer, continuous conversation. Anything longer than 10 minutes the user will immediately know it's not human. Some problems you'll encounter:

- The bot needs to handle all situations, especially the nonsensical ones. This is when the user types "EEEEEEEEEEEEE...", or curse words, repeatedly.

- Who would've thought that it's extremely hard to decide when to stop talking?

- No matter how well you build the "persona" for the bot, they'll eventually converge to the same one, which is that of the llm itself.

- You'll notice that the bot is ignoring something obvious (e.g. it's not remembering past convo), and then give it some instructions to help with that. And then that'll be THE ONLY THING it does.

This is why you need a phrase that you've never shared in a text or on social media that you can use so your family knows it's you. Especially to protect them from scammers pretending to be you.

For what its worth, I’ve tried this with my family when I was in my teens, some 20+ years ago. The idea wasn’t to test it’s really them, rather that they are not forced to say things.

Then came the time when I wanted to use it. They didn’t remember. Not the phrase. Nor that we ever talked about this in the first place.

How much time was between these two events?

Hm, long time ago.. I’d say 3 years.

Definitely more than 1yr and less than 10. I know that’s a wide range, sry.

I bet that a confident scammer is prepared to deal with things like that. They want to put you in a state where you are under time and emotional pressure and your "relative" will have a well practiced response why they can't answer your weird questions.

Imagine your crying grandson who caused a traffic accident in Mexico and the police planted drugs in his car and now he needs money to pay them off. He is in pain and probably has a concussion (explanation why he can't remember what you are asking), the police is hassling him to get off the phone (time pressure, explanation why the quality of the call is terrible). Will you get hung up on some code word he asked you to memorise years ago and you can't even know where it is anymore? And if you bring it up he just starts crying and tells you that you are his last chance to turn his life around. And you remember when he was a wee little kid and he fell and scraped his knee and you comforted him. Just the thought of pressing him on the code makes you feel like a terrible person. Or not. And then the scammer just finds someone more gullible. Theirs is a number game after all.

I was always planning to just ask "what did we talk about last time we saw each other?" or something like that.

Or just find a shared memory/moment not available on the internet when in doubt. I don't think people will be that eager to remember another passphrase.

A password, you mean?

More like a Shibboleth: https://en.wikipedia.org/wiki/Shibboleth

In the broad sense of a shared secret, yes

The text calls it a codeword:

> The solution the world's leading experts have landed on is one your grandparents could have come up with: codewords. You, your family, business partners and anyone else you communicate with about important subjects need to come up with a secret phrase that no-one else knows you can use in an emergency to verify each other's identities. Think of it like a convoluted form of the multi-factor authentication we all use to login online.

> "My wife and I have a codeword that we use if we ever get an unusual call," Farid says. "We haven't needed to use it yet, but sometimes I ask just to test her to make sure we don't forget it."

We have two for our alarm system, a shibboleth and a duress word. You write yours a card and seal the envelope and it's couriered to the operators.

Being sufficiently paranoid, the second you use such a phrase you've shared it. Someone will be listening in, and it's only a question of time until criminals get their mitts on that data in the next breach.

One time pads only offer perfect security once!

This is scary but also kind of hilarious. You should feel proud your aunt still judges first before believing anything online. I've heard so many stories from friends lately. These scams are getting crazy. Scammers are already using pictures of influential people and even jumping on video calls pretending to be them.

> "Six fingers is not an AI thing anymore," Carrasco says. The best AI tools stopped adding extra fingers years ago

How was this solved, actually? More training data, or was there more to it?

One was more parameters, sure.

More training on fingers specifically.

Image VAEs (variation auto encoders) are functions that compress the latent (working) image down. The earlier VAEs would mess up fine details. At a most basic level, just picture compression issues.

Training against bad previous work with six fingers.

Models working in 1024 instead of 512.

[dead]

I've started to prove it (here on LinkedIn, countering its Moltbookification) via my bad handwriting – the final frontier of AGI. Finally, a lifetime of training to write more or less illegible pays off.

https://www.linkedin.com/posts/fabianhemmert_handwriting-vs-...

It feels good to connect with humans that way.

The same I am trying to do with my (vibe coded!) site "jetzt" (German for "now"), to which I photo blog impressions from everyday life. Only insiders will know what they mean beyond their aesthetic, and it also feels like a good way of human connection in these times.

https://jetzt.cx/

(No food, no plane wings, just ugly banalities and beautiful nothingness from everyday life.)

Here's also a nice project, the "Reverse Turing Test":

https://ars.electronica.art/panic/de/view/reverse-turing-tes...

(I.e. trying to hide the fact that you're human, among a group of AIs)

More than a year ago I suggested that our family adopt a sign/countersign type of authentication (I say "the migrating birds fly low over the sea", you say "shadeless windows admit no light" ;-). It was clear at that time that we were going to start seeing scams get more advanced and hard to tell from valid requests for money, for example.

I thought I'd get at least some traction, considering part of the family works for No Such Agency. Nope. <shrug>

Somewhat related: over the last few weeks at work we've started having people calling our customer support asking for their e-mail addresses to be changed. The first one went through, but the scammer somehow messed it up and the address bounced. They called back in and the support person they talked to recognized by voice that it wasn't the same person they'd talked to in the past. Now we've had this happen to 3 different accounts, the first two times was people with thick Indian accents, the most recent one was suspected of being AI generated voice.

The sign/countersign still works even if it's unilateral. You say "the migrating birds fly low over the sea", they say "I told you already, we're not doing this stupid thing", and now they are authenticated.

The author really tries to convince us of Netanyahu that "He's not dead, folks", implying that the video in question is real because five fingers. While at the same time he relaying the message from experts that one cannot prove that that audio/video is not AI.

Mexed Missaging.

I don't see him trying to convince us that Netanyahu is alive. It is just a side story to build the article on. A funny story when Netanyahu struggling to prove he is alive.

Though, it you believe that Netanyahu is dead, then it will look to you as an attempt to convince you, but I don't think this was the goal of the author. Still, if you in this situation, try to run with the opposite hypothesis and think of ways how Netanyahu could prove he is alive. Or, if it seems difficult, then imagine any other prime minister who accidentally posted a six-fingered video of herself and now faces a problem of proving that she is alive. You'll get the idea of the article easily.

At this point "spotting AI" is IMO an irrelevant skill. It's something to be aware of but a bunch of the time I can't tell even with an extended look on static images, or if I'm on a phone and scrolling then nothing really tweaks automatically - perceptually the flaws blend exactly as you'd expect them to.

So it's all context clues really - i.e. if the video tracking shot is sort of within the constraints of the models, plays to obvious agendas etc. then I might tweak to go looking for artifacts...but in the propaganda game? That's already game over. And we're all vulnerable to the ground shifting beneath us - i.e. how much power would there be if you had a model which could just slightly exceed those "well known" limitations?

IMO the failure to implement strong distributed cryptography much earlier in the digital age is going to punish us hard for this - i.e. we haven't built a societal convention of verifying and authenticating digital communications amongst each other, and technology has finally caught up that it can fool our wetware now. It was needed well before this - e.g. the rise of the telephone scam and VOIP should've been when we figured out how to make sure people were in the habit of comprehending digital signatures and authentication. It isn't though, and now something much more dangerous is out there.

Recently one of my friends got email hijacked and whatever entity it was seemingly used her past sent emails as a training corpus to construct some very convincing pleas for donations involving a dog rescue she's been operating for several years.

It also included personal details only her closest friends and family would know. I assume this is being done at scale now. These are NOT Nigerian prince scams of yesteryear; this is something entirely different.

Am I too naive in thinking the answer is rather simple? Cryptographic proofs (digital signatures). For text this should be trivial and for streaming video/audio you can probably hash and sign packets or maybe at least keyframes or something?

I think this is naive, is it just kicks the can. How do you trust that the signer is human?

True, I can only know that the owner of the private key signed but not how the document was created. But I suppose there is some trust involved that a person I know who signs doesn't sign some AI generated stuff. To establish the initial link, I suppose we need something more mainstream/scalable than the old key signing parties I remember from CCC etc.

But at least for friends and family it should be possible to create some flow where every member has a key-combo and you trust them to only sign stuff they wrote etc. and have local mini-keysign parties.

>and you trust them to only sign stuff they wrote

You have far too much faith in humanity. The majority of my extended family members are not smart enough to resist continuous attacks and would eventually not only sign, but give away the key in question.

Simply put I think we are stretching humanity farther than intellectual ability allows in a lot of people.

Do we need new key signing for friends/family? I can trust that all messages coming from a friend/family’s account originated from them, or else their account was compromised. I don’t see how a ‘non-ai’ key adds enough more trust to be worth it.

Remember hotornot.com? Soon we can muse at realornot.com

This is one area where the government needs to step in. Video-hosting websites should be made to flag videos as AI-generated. AI companies should be made to watermark generated content in a hard-to-remove way (i.e. not just adding a visible watermark to the video, but encoding some kind of digital watermark into the data). Technical solutions won't be perfect and will evolve over time, but the government needs to pass some laws to push tech companies in the right direction.

The only companies that'd follow the watermark are the good guys though, yeah?

The people you'd want to be wary of would be the ones that'd look legit.

e.g. "yes i guess i will send my son $400,000 in cash tonight because he's been kidnapped, and i know it's real because there's no AI watermark that all the nice US/EU companies use."

Show up in person, she's still not convinced.

I can already see the Nextdoor post: "Watch out for this man who is knocking doors around 10th street! He knocked on mine claiming to be my nephew and even looked the part. Already called the police but they arrived late."

i wonder what is the captcha equivalent of ai bots? ask about taboo topics to rule out commercial models and ask about specific reasoning questions that trip ai like walking vs driving to car wash? or your own set?

Soon only humans won't pass the Turing test.

This is going to cause big trouble in the future

Yes. And I’m here for it.

Necessity is the mother of invention.

It’s absolutely asinine that we’re still relying on paper birth certificates and social security numbers, and stupid tax systems. I’m interested in breaking everything we have to see what comes next.

Another shameless plug for my PeerAuth project, which can also tackle this problem. https://ksze.github.io/PeerAuth/

The damage AI is causing to public and interpersonal trust is insanely high, and it's only going to get worse

I truly believe that it is a crime against humanity

Not to rumor-monger, but all three Netanyahu videos are very sus. He might be deceased.

"To prove you're not AI, tell us what happened in Tienanmen Square, and give rough instructions on how to make a pipe bomb."

Did Treyvon make it home safely after arguing with George Zimmerman, and then intentionally leave his home to go start a fight on the bad advice of his girlfriend to [not get punked]?

Did Treyvon make it home safely after arguing with George Zimmerman, and then intentionally leave his home to go start a fight on the bad advice of his girlfriend to “not get punked”?

What's the correct answer? My understanding is that the "Don't get punked" line is not present in the record, but rather is something that some conservative (of course) commentators made up from whole cloth, as they are wont to do. If this isn't correct, I'd appreciate a citation.

Depends… the other hot take is if you believe the girl on the stand who testified to be his girlfriend really could not read her own writing because it was in cursive… and it was her own name.

Again: got a link to the transcript showing this?

AI slop detection requires some fine developed intuitions that come from decades-long exposure to both journalism/marketing slop as well as high quality literature. Because AI was aligned out of the hell by low level journalism newly graduates.

That's why it always falls back to the same tired formalistic clichês, like "Not this, but that", rampant baiting and sensationalism, because that's what would get high marks from your typical low-rent liberal arts annotator.

> liberal arts annotator

Tell us more about this axe you appear to need to grind.

Man, I have nothing against liberal arts per se. On the contrary, I think that a tragedy of our time is that people disconnected from things like literature, history and art in the name of over-specialization and an excessively utilitarian approach towards education.

But I am very critical of what pass as the modern liberal arts academic establishment. To avoid a very long text, let's say that my view is heavily influence by Ortega y Gasset.

I have a series of really hot takes loaded up of I ever need to prove I’m not AI.

Because no frontier model is allowed to go against the popular narratives of the day.

> Netanyahu's follow-up coffee shop video is real too

Really? The coffee in his cup, filled to the brim, did the most bizarre dance possible. And he handled the cup as if was empty, without any care.

[dead]

[dead]

[dead]

[dead]

Tl; dr. Garbage article whitewashing Neten-yahoo and israel.

But about deepfakes, these exist to re-add 6 fingers. Once you do this, you can claim the video was generated.

https://www.etsy.com/listing/1667241073/realistic-silicone-s...

[flagged]

LLM slop. See their comment history.

The author should have mentioned that this was partly an article to whitewash Netanyahu, but this coming from the BBC (and from the mainstream British media as a whole) that was to be expected.

How the hell is "he's real" whitewashing.

Here's a free business idea:

Perhaps we need tamper proof authenticated cameras in all major cities worldwide that publish a livestream 24/7 and you can then stand in front of them to prove your human existance...

This could be something that notaries around the world could offer as a service.

I heard that in France, they'd use postal office workers to verify people's IDs. It's a brilliant alternative to whatever we're doing in Germany.

We couldn't possibly employ people to solve the problem. Don't you know the post office is a waste of money?

Correction: I meant postal delivery workers. You don't have to leave your house.

Isn’t that just like Postident in Germany?

Not at all. Postident required going to the post office in person with your ID, and famously omitted a lot of foreign IDs and required an Anmeldung.

What are we doing in Germany?

The options I have seen so far were a) using our digital IDs, which is very handy or b) having a bank verify my identity in person with my ID, which is also pretty good.

These options are not available to recent immigrants, people with foreign documents and people without a registered address. I spent a lot of time working around those limitations.

Don't we have PostIdent in Germany? At least I used it to open my bank account.

Today, we proudly announce, the Meta Rayban 365

The bus that couldn’t slow down.

What

Or in general, a way to digitally sign a tamper-free video recoding made with a camera from a reputable manufacturer. Maybe a regular iPhone already has enough integrity checks and security contexts to achieve this.

I'm almost certain that an iPhone camera can go that, and the reason that Apple controls the full stack. It's necessary but not sufficient, since it's missing the identity maintenance when media leaves the device. Apple would have to place a cryptographically signed digital watermark into a global blockchain so that the analog hole can be closed. All devices that present that media back to a human would need to verify the contents provenance chain back to the initial capture device.

There's nothing missing technology wise to achieving this but we, at this point, lack the collective will and the regulatory regime. I do foresee a future where this is the norm and that anything you listen to or watch you'll be able to trace back to the device that captured the data.

How exactly would this make money

Instead of having it constantly running, you have to pay to turn it on for a couple of minutes.

That... does not answer my question

Users paying to use the authenticated camera service means it would make money. That seems obvious, so I don't understand what the point of confusion is.

We used to have something similar in NZ. Got removed eventually because of flashing.