Author Topic: All Deepfake research must be terminated. Society is not ready.  (Read 2472 times)

0 Members and 1 Guest are viewing this topic.

Offline arthwollipot

  • Reef Tank Owner
  • *********
  • Posts: 9146
  • Observer of Phenomena
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #30 on: July 15, 2019, 10:04:10 PM »
First off: too late. The cat's out of the bag.

Second: who's going to stop Russian and Chinese hackers from accessing the technology illegally? We can't even stop spammers and scammers - this technology is no different.

Third: we've been developing photorealistic CGI for decades now. This is just another step in the same process. A step that uses a new technology, sure, but ray-tracing was a new technology once. No-one's calling for the banning of other CGI technology.
Self-described nerd. Pronouns: He/Him.

Tarvek: There's more to being an evil despot than getting cake whenever you want it.
Agatha: If that's what you think, then you're DOING IT WRONG!

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 8043
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #31 on: July 15, 2019, 10:21:24 PM »
So are you saying that you want to ban research on neural networks and genetic algorithms?
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #32 on: July 15, 2019, 10:56:30 PM »
You guys are right... there is no stopping it...

Understanding the horrible and destructive ways in which this technology will be used, I just felt that something should be done...

Maybe not even the Pentagon would be able to do anything about it...
"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

Offline Captain Video

  • Superhero of the Silver Screen
  • Frequent Poster
  • ******
  • Posts: 3576
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #33 on: July 16, 2019, 01:26:19 AM »
I still don't think this is as undetectable as you think.

As far as face swapping or any swapping you have to match camera, lens, lighting, pixel size, artifacting and many other elements not to mention additional compositing.  I don't believe this will be "pixel perfect" or otherwise undetectable in the next 10 years and would be spotted by simple eye examination by a professional who could prove exactly why the video is a fake.

This would also be true of any deep fake regardless of type where the audio was altered. We have specialists who can analyze this already as the ability to manipulate audio has been almost perfect and undetectable for an untrained ear for some time now. People don't get away with it because when you look at the waveform it becomes obvious that the audio is fake or manipulated in some way.

There are some concerns, as you explained its going to be easier to fool your neighbor as well as the average FB user with video and people are going to have to be more skeptical about it. Just like memes and fake news, some people will believe the fakes even when you do prove them incorrect, whats new?

I cant wait for  "Penn and Tellers "Fuck with your friends with deep fakes app"   
“Don't explain computers to laymen. Simpler to explain sex to a virgin.”
― Robert A. Heinlein, The Moon is a Harsh Mistress

Offline John Albert

  • Too Much Spare Time
  • ********
  • Posts: 6841
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #34 on: July 16, 2019, 05:16:57 AM »
Another thing to think about is that the "deep fake" videos typically involve compositing of two or more images. That means that the originals had to come from somewhere.

For example if you're altering a video of a president giving a speech, you'd probably have to start with some source material that is probably already in the public domain, and that video might have some recognizable elements that indicate its location and time.

So even if the illusion is rendered so convincingly that the difference cannot be discerned through casual viewing, certain clues might be detectable that give away the identity of the original source material.

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #35 on: July 16, 2019, 06:53:29 AM »
Guys, check this out: this is the level of perfection that the AIs can reach when faking voices. I don't know about the audio spectre analysis... but to the human ears, I'd say the technology today already is 95% of the way.

"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

Offline stands2reason

  • Empiricist, Positivist, Militant Agnostic
  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 10746
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #36 on: July 16, 2019, 10:59:26 AM »
Second: who's going to stop Russian and Chinese hackers from accessing the technology illegally? We can't even stop spammers and scammers - this technology is no different.

Exactly. If you criminalize deepfakes, only criminals will continue to use deepfakes. We should just voluntary stop technical advancement and let other countries take the tech lead instead...

Offline Captain Video

  • Superhero of the Silver Screen
  • Frequent Poster
  • ******
  • Posts: 3576
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #37 on: July 16, 2019, 12:53:28 PM »
Another thing to think about is that the "deep fake" videos typically involve compositing of two or more images. That means that the originals had to come from somewhere.

For example if you're altering a video of a president giving a speech, you'd probably have to start with some source material that is probably already in the public domain, and that video might have some recognizable elements that indicate its location and time.

So even if the illusion is rendered so convincingly that the difference cannot be discerned through casual viewing, certain clues might be detectable that give away the identity of the original source material.
\
thats my point with the pixels. The public domain video is most likely under 1080p and already encoded for the internet, likely using an H264 codec or similar. Just using the face pixels reduces this to an even smaller size, then you compost that face onto a different video. No matter how good it looks on the final composite if you zoom in to pixel size there will be a noticeable difference between the two sets of pixels.

garbage in = garbage out.

I'm not saying you cant composite with h264 but it would make it much harder to make it look good and it wont be pixel perfect, regardless of how much time was spent on it.

if you are starting with two prores 444 or uncompressed videos that were shot on the same camera with the same lens and the same lighting maybe but even then you would notice subtle differences in the composite when zooming in.
“Don't explain computers to laymen. Simpler to explain sex to a virgin.”
― Robert A. Heinlein, The Moon is a Harsh Mistress

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #38 on: July 16, 2019, 01:07:55 PM »
Am I being unreasonable for freaking out because of this? Am I wrong to state that this will forever change our society in a very profound, permanent and negative way?


People are already pointing out the huge danger of that Joe Rogan AI... Someone could get a phone number of someone he knows... Use this AI to say "Let's meet at the street X at 11pm".... and fucking kidnap the person! Or call one of his businesses, act like himself, and then extract confidential information!

I understand that the development of this technology cannot be stopped... So what could we do instead of just sit and wait?
"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #39 on: July 16, 2019, 01:20:44 PM »
@Captain Video

I think your reasoning is being affected by your Expert bias. You understand all the tools and technologies, and it is extremely easy for you to spot it. I think your bias is preventing you to accurately assess how perfectly real this will look like for the vast majority of the population. You keep bringing up analysis techniques, but that will be completely irrelevant in many, many cases. Think about how many videos you watched in the last month. How many of them did you bother to verify? Like I've said, the answer to the question if it is fake or not can be on the first link of a google search... but the vast majority of the population will not be reached by it.

We must find ways to protect ourselves from the vast damage that this technology will bring... and the solution will not simply be to create detection algorithms. My point is, the detection algorithm can be technically effective... but in reality, SOCIALLY, it will not.
"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

Offline Captain Video

  • Superhero of the Silver Screen
  • Frequent Poster
  • ******
  • Posts: 3576
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #40 on: July 16, 2019, 01:35:37 PM »
@Captain Video

I think your reasoning is being affected by your Expert bias. You understand all the tools and technologies, and it is extremely easy for you to spot it. I think your bias is preventing you to accurately assess how perfectly real this will look like for the vast majority of the population. You keep bringing up analysis techniques, but that will be completely irrelevant in many, many cases. Think about how many videos you watched in the last month. How many of them did you bother to verify? Like I've said, the answer to the question if it is fake or not can be on the first link of a google search... but the vast majority of the population will not be reached by it.

We must find ways to protect ourselves from the vast damage that this technology will bring... and the solution will not simply be to create detection algorithms. My point is, the detection algorithm can be technically effective... but in reality, SOCIALLY, it will not.

I do think you are being unreasonable with your fears

I hear your concern but again this is nothing new.  Just like plenty of people are fooled by Photoshop images they will also be fooled by video, hell people are fooled by video now without the deep fakes.

You often don't need to be an expert when it comes to being skeptical about something.

Someone plays you a video of your wife doing the neighbor, "well that cant be right, perhaps I should check into this before making any rash decisions."  Wife: "thats not me its faked,  I was at work"  no need to analyze the video when you can confirm with a phone call and call her boss (you should believe your wife in the first place)

The more it happens the less people will be fooled by it.
“Don't explain computers to laymen. Simpler to explain sex to a virgin.”
― Robert A. Heinlein, The Moon is a Harsh Mistress

Offline Captain Video

  • Superhero of the Silver Screen
  • Frequent Poster
  • ******
  • Posts: 3576
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #41 on: July 16, 2019, 01:48:46 PM »
BTW,  No one has ever actually called me an expert before, especially in this place, you have done this twice now.  It kind of feels really good.

Seriously, thank you for that.  And thank you for treating this as a conversation and not a war.  I absolutely do see and understand many of your concerns. I agree this is not something to be completely ignored. Skeptics will have to stay on their toes.


“Don't explain computers to laymen. Simpler to explain sex to a virgin.”
― Robert A. Heinlein, The Moon is a Harsh Mistress

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 8043
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #42 on: July 16, 2019, 02:35:15 PM »
When I am considering whether it is reasonable to ‘freak out’ about something, I always consider whether there is anything I can do about it. If there is, then there is no reason to ‘freak out’ about it; i should just do what I can to fix it. If there is not, then there is no point in ‘freaking out’ about it, since doing so will not improve the situation.  In either case, freaking out is the wrong approach.  And so is calling for the government to ban basic computer science research out of fear.
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #43 on: July 16, 2019, 07:30:00 PM »
When I am considering whether it is reasonable to ‘freak out’ about something, I always consider whether there is anything I can do about it. If there is, then there is no reason to ‘freak out’ about it; i should just do what I can to fix it. If there is not, then there is no point in ‘freaking out’ about it, since doing so will not improve the situation.  In either case, freaking out is the wrong approach.  And so is calling for the government to ban basic computer science research out of fear.

Well, I might have exaggerated my choice of words a bit. I used the expression "freaking out" only to explicit my levels of concern against the perceived calmness that I've sensed here.
You've spoken like a Stoic. Wise words indeed. I've finished Marcus Aurelius Meditations and I'm starting Epictetus letters now. I'm already habituating myself to live by that attitude.
As for the call for the ban on this research.... Yeah, after starting this topic it quickly become evident to me that that is not feasible.

BTW,  No one has ever actually called me an expert before, especially in this place, you have done this twice now.  It kind of feels really good.
Seriously, thank you for that.  And thank you for treating this as a conversation and not a war.  I absolutely do see and understand many of your concerns. I agree this is not something to be completely ignored. Skeptics will have to stay on their toes.

Well, I'm so glad that this discussion is heading this way... when there is an absolute separation between our arguments and ourselves, is when all of us learn and adapt faster. :D
"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

Offline John Albert

  • Too Much Spare Time
  • ********
  • Posts: 6841
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #44 on: July 16, 2019, 07:42:18 PM »
Am I being unreasonable for freaking out because of this?

No, you are not. This is legitimately freaky. But as The Latinist suggested, holding your emotions in check is a major component of rationality. 

'Deep fakes' are an emerging, revolutionary technology that has the potential to create convincing false narratives by literally putting words into the mouths of prominent leaders. As such, it has the potential to undermine our perceptions of reality in ways we cannot even imagine at this point. There's no telling the level of damage it could cause.


Am I wrong to state that this will forever change our society in a very profound, permanent and negative way?

That remains to be seen, but this is just another step in a destabilization process that is already well underway. For the past couple decades we've been dealing with an erosion of confidence in facts and reality, to the point where our trust in information sources has become practically arbitrary. In this environment, the side of honesty, transparency and science is never going to win out by merely pitting narrative against narrative.

What we need to do is create a new culture that values facts over bullshit. We can start by educating people on the importance of approaching all information in a skeptical way, and empowering everybody with the tools of critical thinking and the techniques for identifying deception.

For one thing, we need to start paying attention to information instead of investing so much confidence in personalities.
« Last Edit: July 16, 2019, 07:52:52 PM by John Albert »

 

personate-rain