Author Topic: All Deepfake research must be terminated. Society is not ready.  (Read 2473 times)

0 Members and 1 Guest are viewing this topic.

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #15 on: July 14, 2019, 07:25:56 PM »
@Captain Video, thanks for your input, it is greatly appreciated to have an expert opinion on this discussion.

Quote
Edit: "Deep fake" is just a media buzz word for what we call compositing.

The term Deepfake is not just a media thing. Semantically, it is very relevant, as it separates the two video editing techniques: Compositing is something that requires way more expertise to be done realistically, and takes longer to render. The term Deepfake comes from "Deep Learning AI", and achieves better results faster. Traditional composition techniques do not employ deep learning, and therefore, are not "deep". The word Deepfake is meaningful and useful.

I understand that you are telling me that the same result can also be achieved with current composition techniques. I'm not worried about that. My concern comes with the astronomical increase of both realism and availability that this new technology will bring to society.

If you composite Trumps face onto a pig... 100% of people will dismiss it as obviously fake, and the impact on the truth would be precisely zero.

Let me try to be more clear about my concern:

Imagine the scenario that an uneducated person in a suburban area wants to mess with one of the neighbors that he dislikes. He get hold of a video of that guy speaking innocent things at a random party. He then downloads an app on his crappy cellphone that allows him to "Rewrite videos by editing text". He then changes the sentence "Thank you guys... I had a lot of fun with you tonight. We should do this more often..." to  >>>  "Guys.... I've just fucked X's wife really hard... I've throat-fucked that whore...". The video is sent to X. Overpowered by rage and hate, he gets his gun and walks towards the guys house, opens the door, and instantaneously kill the guy.

Would composition achieve the same result in this case? NO! The guy wouldn't have the energy and talent to invest many hours in becoming expert on the Adobe Creative Suite. This technology would allow ANYONE to create PERFECT FAKES, IN REAL TIME, WITH CRAPPY CELLPHONES!

It would be impossible to verify them all. In many, many cases irreversible damages would be done within hours of targeted people watching the fake video.

This technology would bring an unprecedented massification of "alternative-facts" to society. We are going to be living inside some real Black Mirror shit....
"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

Offline John Albert

  • Too Much Spare Time
  • ********
  • Posts: 6841
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #16 on: July 14, 2019, 07:43:55 PM »
There's no putting the genie back into the bottle.

We need some kind of digital signing system for all photo, audio and video.

MD5 hashes have long been used by software repositories and file sharing networks to establish authenticity. MD5 is not perfect; collisions (where two different files result in the same hash value) can be caused by adding extraneous junk data. But doing so would drastically change the file size, so knowing a combination of the file size and MD5 hash value is a very good indicator of authenticity.

The problem is, most credulous people and motivated ideologues can't even be bothered to do even the bare minimum of fact-checking, so it's probably too much to expect them to bother tracking down the source of a given video and comparing file sizes and hash values.

Offline Captain Video

  • Superhero of the Silver Screen
  • Frequent Poster
  • ******
  • Posts: 3576
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #17 on: July 14, 2019, 08:01:41 PM »
sorry my friend, its the same argument we hear every time some new tech comes out.  Life can be manipulated.

they said it with photography and people made fakes (UFOs, Bigfoot, etc), they said it with photocopiers and people made fakes (watch Better Call Saul) , they said it with photoshop and people made fakes (every meme on the internet), video is no different.

think of it as 24-60 photoshop comps per second.  New software only allows people to change things faster.

The video example you posted would be impossible without those compositing techniques.  Not to mention the shooting, lighting, and editing. I took one look at that video and instantly knew it was fake despite the excellent job done by the creators.

Did you notice the actual pixel size of the face? They discuss how small that is along with the limitations. Had they not shot it a certain way with a similar face they could not have done it.  They needed a crew. Most importantly you cant pull off the "deep fake" without also using those compositing techniques. The software does not do the compositing for you.

In the situation you describe above one guy is a murderer who obviously cant take a joke, the other guy is not at fault legally. You would have to prove that the image manipulator had malicious intent and he may not have. Someone that is better with US law would be able to describe why better than I can. Latinist?

You could probably call me an expert because of my job and experience but actual video forensic experts exist now who already specialize in this sort of thing.
“Don't explain computers to laymen. Simpler to explain sex to a virgin.”
― Robert A. Heinlein, The Moon is a Harsh Mistress

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #18 on: July 14, 2019, 08:25:53 PM »
Quote
The video example you posted would be impossible without those compositing techniques.  Not to mention the shooting, lighting, and editing. I took one look at that video and instantly knew it was fake despite the excellent job done by the creators

I am talking about the technique on the first video of this topic, that allows anyone to change a real video by editing text, not the Keanus one.

Quote
In the situation you describe above one guy is a murderer who obviously cant take a joke, the other guy is not at fault legally. You would have to prove that the image manipulator had malicious intent and he may not have. Someone that is better with US law would be able to describe why better than I can. Latinist?

Well, in my example the guy got enraged not because in his head "he cant take a joke". In his head, that guy literally had sex with his wife and was bragging about it. In his head, it was 100% real.

You raised an interesting point. How does American Law would react in this case? Do any laws exists that would punish the guy who created the fake video that was the motivation of a murder?
"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

Offline Captain Video

  • Superhero of the Silver Screen
  • Frequent Poster
  • ******
  • Posts: 3576
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #19 on: July 14, 2019, 08:38:13 PM »
Just to be clear on my last comment

Quote
you could probably call me an expert because of my job and experience but actual video forensic experts exist now who already specialize in this sort of thing.

I'm saying that I don't have the experience of a video forensics expert but with the limited experience in my field Im pretty good at spotting VFX. I looked one up out of curiosity and found some interesting information on how they investigate video.

here is one at random from Detroit.

http://www.videoforensicexpert.com/video-authentication-analysis-edit-detection-and-tampering/

Quote
We begin all forensic video authentication services and analysis investigations with a preliminary analysis.  Throughout this authentication process, our experts review the video recording in depth through frame by frame analysis. The goal of the preliminary analysis is to determine if any signs of tampering, anomalies or other red flags are present.

I don't think proving a deep fake is any different than normal video analysis. At this point its probably even easier to spot than other methods of fakery which already have a system of investigation in place.
“Don't explain computers to laymen. Simpler to explain sex to a virgin.”
― Robert A. Heinlein, The Moon is a Harsh Mistress

Offline John Albert

  • Too Much Spare Time
  • ********
  • Posts: 6841
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #20 on: July 15, 2019, 01:35:57 PM »
I think we already had a case similar to this

https://en.wikipedia.org/wiki/Hustler_Magazine_v._Falwell

Was that the trial where Larry Flynt showed up in court wearing a diaper made out of the American flag and a baby bib with a Nazi medal pinned on?

Offline Captain Video

  • Superhero of the Silver Screen
  • Frequent Poster
  • ******
  • Posts: 3576
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #21 on: July 15, 2019, 01:57:10 PM »
I think we already had a case similar to this

https://en.wikipedia.org/wiki/Hustler_Magazine_v._Falwell

Was that the trial where Larry Flynt showed up in court wearing a diaper made out of the American flag and a baby bib with a Nazi medal pinned on?

no this was the supreme court case where he allegedly behaved himself.
“Don't explain computers to laymen. Simpler to explain sex to a virgin.”
― Robert A. Heinlein, The Moon is a Harsh Mistress

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 8043
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #22 on: July 15, 2019, 02:04:30 PM »
I’d still like to hear what exactly the OP wants to ban.  I suspect he has no real idea himself.
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline John Albert

  • Too Much Spare Time
  • ********
  • Posts: 6841
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #23 on: July 15, 2019, 02:44:04 PM »
I'd like to know who he expects to institute and enforce this ban, along with a description of the investigative and punitive practices he expects those authorities to take.
« Last Edit: July 15, 2019, 03:42:59 PM by John Albert »

Offline Captain Video

  • Superhero of the Silver Screen
  • Frequent Poster
  • ******
  • Posts: 3576
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #24 on: July 15, 2019, 04:40:49 PM »
I think we already have the laws in place to stop the criminal aspect of this and he may not in his country.
“Don't explain computers to laymen. Simpler to explain sex to a virgin.”
― Robert A. Heinlein, The Moon is a Harsh Mistress

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #25 on: July 15, 2019, 06:05:54 PM »
I’d still like to hear what exactly the OP wants to ban.  I suspect he has no real idea himself.

Sorry, my bad. I focused so much on the other points raised that I've forgotten to answer you.

Well, I just think this "ban" would mean a temporary hold on creating and sending money to start new research on this area... Or maybe a summit with scientists from this area to discuss it further in detail before continuing. I think everybody should pause for a year or 2 to calm down, and understand the real impacts and consequences of this. This literally will change forever our perception of reality... its too disruptive to be simply treated as any other study.

You guys can be 100% sure. This technology WILL be weaponized. Imagine people in 2025, scrolling their Facebook timelines, and encountering many videos. One of them is a pixel perfect fake, with the face of a very important scientist saying "Well, it was proven that vaccines do cause autism" or "Climate change is not a thing."

And our perception from the past will change too! We will start to see videos stating that they represent history... but they are completely fake.

Its more than obvious that a very cautious analysis must take place.
"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

Offline John Albert

  • Too Much Spare Time
  • ********
  • Posts: 6841
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #26 on: July 15, 2019, 06:46:52 PM »
I don't disagree that this tech is going to be used for some fucked-up purposes. I'm just wondering what can be done about it.

There are no gatekeepers to the development this kind of technology, so it's not like a "summit" of scientists can really accomplish anything. Some developers work on these kinds of projects just out of personal interest. I would imagine that most of the research grants for this kind of thing (if such grants exist) are coming through an intelligence or defense-related agency like DARPA.

Offline Captain Video

  • Superhero of the Silver Screen
  • Frequent Poster
  • ******
  • Posts: 3576
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #27 on: July 15, 2019, 06:50:32 PM »
I’d still like to hear what exactly the OP wants to ban.  I suspect he has no real idea himself.

Sorry, my bad. I focused so much on the other points raised that I've forgotten to answer you.

Well, I just think this "ban" would mean a temporary hold on creating and sending money to start new research on this area... Or maybe a summit with scientists from this area to discuss it further in detail before continuing. I think everybody should pause for a year or 2 to calm down, and understand the real impacts and consequences of this. This literally will change forever our perception of reality... its too disruptive to be simply treated as any other study.

You guys can be 100% sure. This technology WILL be weaponized. Imagine people in 2025, scrolling their Facebook timelines, and encountering many videos. One of them is a pixel perfect fake, with the face of a very important scientist saying "Well, it was proven that vaccines do cause autism" or "Climate change is not a thing."

And our perception from the past will change too! We will start to see videos stating that they represent history... but they are completely fake.

Its more than obvious that a very cautious analysis must take place.

How would you enforce that? The code for some of this software (again the keanu video) is available on github which makes me doubt that any money is being spent on "research" unless they are doing it for a private company making entertainment based software like Adobe or Autodesk.

I also have extreme doubts that a "pixel perfect"  image will ever be available, even in the next 5 years.
“Don't explain computers to laymen. Simpler to explain sex to a virgin.”
― Robert A. Heinlein, The Moon is a Harsh Mistress

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #28 on: July 15, 2019, 07:20:13 PM »
The Pentagon is already considering this technology as a threat. Also, the 2020 election will be targeted.

"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

Offline Igor SMC

  • Keeps Priorities Straight
  • ***
  • Posts: 328
Re: All Deepfake research must be terminated. Society is not ready.
« Reply #29 on: July 15, 2019, 09:07:56 PM »
@Captain Video
This is where the word "Deep" comes from. This is why this is fundamentally different from usual compositing... And the most worrying part? Deepfakes uses Generative Adversarial Networks. Did some genius developed an algorithm that is able to detect fake videos? A person with little knowledge of programming will simply copy that code, and USE IT as the Discriminator. The AI generates an image. Can the Discriminator detect that is fake? Change parameters and try again... After some iterations, the generator will create a frame, and the Discriminator will tell it is real... And so the next frame in the video is created. By design, this algorithm WILL ALWAYS generate fakes that the detection algorithms failed to flag. Also, keep in mind that another layer of AI is used that improves each guess for the frame. This means that for the first frames the Generator will fail to evade the Discriminator in many iterations... but after a while, it will learn how to efficiently avoid the Discriminator with less iterations. Deepfakes, by design, will always be one step ahead....  :-\

"Knowledge is preferable to ignorance. Better by far to embrace the hard truth than a reassuring fable"

 

personate-rain