Skeptics Guide to the Universe Forums

General Discussions => Skepticism / Science Talk => Topic started by: Igor SMC on July 14, 2019, 11:07:56 AM

Title: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 14, 2019, 11:07:56 AM
Rewrite Videos By Editing Text
https://www.youtube.com/watch?v=2xWnOL5bts8

https://www.youtube.com/watch?v=aJq6ygTWdao


Deepfake research must be ruled unethical for the moment. I know that it may sound extreme at first, but when you stop to analyze the situation from a realistic point of view, this becomes very clear. What are the alleged benefits of this technology? Entertainment. What are the potential damages? Being weaponized for political purposes. During the grey zone that we are going to through in which this is a new thing, to the moment society and our laws are somewhat adapted to it, innocent people will get incriminated by false videos. Legitimate videos will be discarded as evidence by someone stating that it was a deepfake. At a certain point, and this is inevitable, video will no longer be considered strong evidence in investigations and will achieve the same level of reliance as a witness account that must be investigated.

People are quick to point out that the technology to detect these fakes are also being developed... But to say this, is to completely ignore how misinformation spreads through the internet and social circles. Someone will create a pixel perfect deepfake. Another person will use algorithms to prove it is fake. The fake version will spread exponentially on the internet, and by people speaking to one another physically. Even if the real video is one google search away, the vast majority of people will not be reached by it. At that moment, it could be too late. An election day might have passed. A person lost his job. Someone's reputation completely obliterated. Someone being lynched and killed for a crime he did not commit... and the benefits of this technology? Entertainment...
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 14, 2019, 11:23:28 AM
so you want us to get rid of all green screens, rotoscoping, mo-cap and compositing or just the new adobe plug-in that saves time and money by doing it quickly?

Should we ditch Photoshop?

I cant agree with you.


Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Soldier of FORTRAN on July 14, 2019, 11:51:58 AM
There's no putting the genie back into the bottle.

We need some kind of digital signing system for all photo, audio and video.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 14, 2019, 12:03:11 PM
so you want us to get rid of all green screens, rotoscoping, mo-cap and compositing or just the new adobe plug-in that saves time and money by doing it quickly?

Should we ditch Photoshop?

I cant agree with you.

Rotoscoping, mo-cap and compositing are expensive things that pose very little danger of misuse, and require specialists and hardware to create. This technology is cheap and fast, hundreds of millions of common people would be able to use it. This will cause a real flood of fake videos.

About Photoshop: In investigations digital images are already analyzed with skepticism because of it. Thats why they hire specialists in image manipulation before validating it as evidence.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 14, 2019, 12:17:56 PM
so you want us to get rid of all green screens, rotoscoping, mo-cap and compositing or just the new adobe plug-in that saves time and money by doing it quickly?

Should we ditch Photoshop?

I cant agree with you.

Rotoscoping, mo-cap and compositing are expensive things that pose very little danger of misuse, and require specialists and hardware to create. This technology is cheap and fast, hundreds of millions of common people would be able to use it. This will cause a real flood of fake videos.

About Photoshop: In investigations digital images are already analyzed with skepticism because of it. Thats why they hire specialists in image manipulation before validating it as evidence.

all Adobe software for $50 a month, $25 if you have a student ID, the demo works for 30 days and there are a million cracks available.

With After effects I can now remove you from a video with the click of a button and no green screen.

All of the high end compositing software is free, you can download it right now and get to work, you only pay if you use it to make money. Most "specialists" got that way by downloading it and playing with it until they got it right.

As to hardware your average computer can handle all of this. It just takes longer to render.

I agree we are about to see a flood in video fakes but just like we did with Photoshop we will need experts to analyze the videos.  Stopping the advance of technology is not the answer.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Awatsjr on July 14, 2019, 01:32:52 PM
I can’t swim.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 14, 2019, 01:41:01 PM
The João Doria Case

João Doria is one of the most influential politicians in Brazil. He was the mayor of São Paulo, the largest and most populated city in the southern hemisphere of the globe. 5 days prior to the election, a video surfaced alleging that he was using public money to have orgies with several prostitutes. He claimed that the video was "obviously a deepfake". On youtube and several articles on the internet it was being claimed that the video was analyzed by experts, and it was concluded it was indeed a fake... The next day, other videos of other experts appear claiming that it was real. Now... how can the public judge in this case? Since "experts" were giving contradictory statements, who can we trust? Should we believe that he bribed some experts to say that it was a fake, or should we believe that someone from the opposition bribed someone to create the video and claim that it was real?

In both cases, my point in this topic is explicitly proven.

He is innocent and is suffering damage to his public image because of the video? Deepfakes are the problem.
The video is legitimate and he managed to get away from it by simply saying it was a fraud? Deepfakes are the problem.

(https://www.tocantinsagora.com.br/wp-content/uploads/2018/10/doria_orgia.jpg)

Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 14, 2019, 01:53:26 PM
I don’t think there is any way to put this genie back in the bottle.  There may be a way of hashing or signing images to make it easier to detect modification, but ultimately we’re going to have to get used to the fact that seeing just isn’t believing any more. This generation has already dealt with the death of the expectation of privacy, and the next will have to deal with this.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 14, 2019, 02:00:27 PM
Keanu Reeves Stops A ROBBERY!
https://www.youtube.com/watch?v=3dBiNGufIJw

This technology allows someone to create an extremely realistic looking image of Keanu Reeves doing something nice? This technology allows the creation of a career-destroying video of Keanu Reeves too.

You mentioned other techniques... but they are very, very far from creating something so realistic, with so little expertise and resources. After Affects and Premiere Pro algorithms are not neural networks!!! This is absolutely a different level!

Imagine the scenario where someone uses actors and this technology to accuse someone of rape! Imagine how exponentially fast the fake video will spread, and how much time, effort and money will it take for the person to spread the truth to 10% of that same group of people...
At the same time, imagine that a real rape video surfaces, and the criminal getting away simply by saying it was a deepfake!

Also, keep in mind the difference in quality of deepfakes of 2019 from the ones in 2015. Now extrapolate this to deepfakes of 2025. Do you see how destructive this technology is??? For what?? Just for the sake of entertainment???
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 14, 2019, 02:53:40 PM
What exactly do you think such a ban would entail?
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 14, 2019, 03:05:52 PM
What exactly do you think such a ban would entail?

First, allow time for legislation to be created... Celebrities must have the right to sue anyone creating deepfakes with their faces. They are by far the most vulnerable, since it's very easy to create perfect fakes... AI systems need material to learn... and it's very easy to find literally thousands of images and hundreds of hours of video of them.

Second, allow scientists more time to deliberate and understand the real consequences. All fields of science must respect ethical barriers of research... why in the digital realm it should be a free for all?

Legislation must also be created to increase the punishment of people using deepfakes as evidence...

This is what I mean by the fact that objectively society is not ready. We don't have the culture, the technology or the legislation yet to cope with something so extremely serious like this.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 14, 2019, 03:36:28 PM
You have not explained what a ban on research would consist of.  “Allowing time” is a goal, not a method.  So what research techniques, specifically, would you ban and how would you enforce it? What secondary effects and unintended consequences do you anticipate, and how would you mitigate them? What about dual-use technologies?
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 14, 2019, 04:10:07 PM
What exactly do you think such a ban would entail?

First, allow time for legislation to be created... Celebrities must have the right to sue anyone creating deepfakes with their faces. They are by far the most vulnerable, since it's very easy to create perfect fakes... AI systems need material to learn... and it's very easy to find literally thousands of images and hundreds of hours of video of them.

Second, allow scientists more time to deliberate and understand the real consequences. All fields of science must respect ethical barriers of research... why in the digital realm it should be a free for all?

Legislation must also be created to increase the punishment of people using deepfakes as evidence...

This is what I mean by the fact that objectively society is not ready. We don't have the culture, the technology or the legislation yet to cope with something so extremely serious like this.

wait, so I cant composite Trumps face on porky pigs face wearing a Nazi uniform without him suing me?

I think we already had a case similar to this

https://en.wikipedia.org/wiki/Hustler_Magazine_v._Falwell

It would already be against the law to composite his face onto (another) criminals face in a video then try to frame him for the same crime claiming it was him.  I don't know what would need to change.

To do so for entertainment purposes is not, nor should it be against the law.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 14, 2019, 04:22:55 PM
Keanu Reeves Stops A ROBBERY!
https://www.youtube.com/watch?v=3dBiNGufIJw

This technology allows someone to create an extremely realistic looking image of Keanu Reeves doing something nice? This technology allows the creation of a career-destroying video of Keanu Reeves too.

You mentioned other techniques... but they are very, very far from creating something so realistic, with so little expertise and resources. After Affects and Premiere Pro algorithms are not neural networks!!! This is absolutely a different level!

Imagine the scenario where someone uses actors and this technology to accuse someone of rape! Imagine how exponentially fast the fake video will spread, and how much time, effort and money will it take for the person to spread the truth to 10% of that same group of people...
At the same time, imagine that a real rape video surfaces, and the criminal getting away simply by saying it was a deepfake!

Also, keep in mind the difference in quality of deepfakes of 2019 from the ones in 2015. Now extrapolate this to deepfakes of 2025. Do you see how destructive this technology is??? For what?? Just for the sake of entertainment???

what do you think this video was made with?  some fancy new "Deep Fake" software?  nope, they used after effects and a green screen

https://www.youtube.com/watch?v=IzEFnbZ0Zd4


Edit: "Deep fake" is just a media buzz word for what we call compositing.

Big edit: Sorry, actually I was wrong, they did use some specialized software in conjunction with AE but I think my point still stands. look how much work it took and how much they needed the other software to sell the effect. They didnt need the extra software and still could have done it with AE or better yet NUKE



Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Gigabyte on July 14, 2019, 07:18:17 PM
Has anyone verified this is actually true?  And if so, how can we know if it is?  It's a very old issue.

How do you know if something is true if you didn't see it with your own eyes?

Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 14, 2019, 07:25:56 PM
@Captain Video, thanks for your input, it is greatly appreciated to have an expert opinion on this discussion.

Quote
Edit: "Deep fake" is just a media buzz word for what we call compositing.

The term Deepfake is not just a media thing. Semantically, it is very relevant, as it separates the two video editing techniques: Compositing is something that requires way more expertise to be done realistically, and takes longer to render. The term Deepfake comes from "Deep Learning AI", and achieves better results faster. Traditional composition techniques do not employ deep learning, and therefore, are not "deep". The word Deepfake is meaningful and useful.

I understand that you are telling me that the same result can also be achieved with current composition techniques. I'm not worried about that. My concern comes with the astronomical increase of both realism and availability that this new technology will bring to society.

If you composite Trumps face onto a pig... 100% of people will dismiss it as obviously fake, and the impact on the truth would be precisely zero.

Let me try to be more clear about my concern:

Imagine the scenario that an uneducated person in a suburban area wants to mess with one of the neighbors that he dislikes. He get hold of a video of that guy speaking innocent things at a random party. He then downloads an app on his crappy cellphone that allows him to "Rewrite videos by editing text". He then changes the sentence "Thank you guys... I had a lot of fun with you tonight. We should do this more often..." to  >>>  "Guys.... I've just fucked X's wife really hard... I've throat-fucked that whore...". The video is sent to X. Overpowered by rage and hate, he gets his gun and walks towards the guys house, opens the door, and instantaneously kill the guy.

Would composition achieve the same result in this case? NO! The guy wouldn't have the energy and talent to invest many hours in becoming expert on the Adobe Creative Suite. This technology would allow ANYONE to create PERFECT FAKES, IN REAL TIME, WITH CRAPPY CELLPHONES!

It would be impossible to verify them all. In many, many cases irreversible damages would be done within hours of targeted people watching the fake video.

This technology would bring an unprecedented massification of "alternative-facts" to society. We are going to be living inside some real Black Mirror shit....
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 14, 2019, 07:43:55 PM
There's no putting the genie back into the bottle.

We need some kind of digital signing system for all photo, audio and video.

MD5 hashes have long been used by software repositories and file sharing networks to establish authenticity. MD5 is not perfect; collisions (where two different files result in the same hash value) can be caused by adding extraneous junk data. But doing so would drastically change the file size, so knowing a combination of the file size and MD5 hash value is a very good indicator of authenticity.

The problem is, most credulous people and motivated ideologues can't even be bothered to do even the bare minimum of fact-checking, so it's probably too much to expect them to bother tracking down the source of a given video and comparing file sizes and hash values.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 14, 2019, 08:01:41 PM
sorry my friend, its the same argument we hear every time some new tech comes out.  Life can be manipulated.

they said it with photography and people made fakes (UFOs, Bigfoot, etc), they said it with photocopiers and people made fakes (watch Better Call Saul) , they said it with photoshop and people made fakes (every meme on the internet), video is no different.

think of it as 24-60 photoshop comps per second.  New software only allows people to change things faster.

The video example you posted would be impossible without those compositing techniques.  Not to mention the shooting, lighting, and editing. I took one look at that video and instantly knew it was fake despite the excellent job done by the creators.

Did you notice the actual pixel size of the face? They discuss how small that is along with the limitations. Had they not shot it a certain way with a similar face they could not have done it.  They needed a crew. Most importantly you cant pull off the "deep fake" without also using those compositing techniques. The software does not do the compositing for you.

In the situation you describe above one guy is a murderer who obviously cant take a joke, the other guy is not at fault legally. You would have to prove that the image manipulator had malicious intent and he may not have. Someone that is better with US law would be able to describe why better than I can. Latinist?

You could probably call me an expert because of my job and experience but actual video forensic experts exist now who already specialize in this sort of thing.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 14, 2019, 08:25:53 PM
Quote
The video example you posted would be impossible without those compositing techniques.  Not to mention the shooting, lighting, and editing. I took one look at that video and instantly knew it was fake despite the excellent job done by the creators

I am talking about the technique on the first video of this topic, that allows anyone to change a real video by editing text, not the Keanus one.

Quote
In the situation you describe above one guy is a murderer who obviously cant take a joke, the other guy is not at fault legally. You would have to prove that the image manipulator had malicious intent and he may not have. Someone that is better with US law would be able to describe why better than I can. Latinist?

Well, in my example the guy got enraged not because in his head "he cant take a joke". In his head, that guy literally had sex with his wife and was bragging about it. In his head, it was 100% real.

You raised an interesting point. How does American Law would react in this case? Do any laws exists that would punish the guy who created the fake video that was the motivation of a murder?
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 14, 2019, 08:38:13 PM
Just to be clear on my last comment

Quote
you could probably call me an expert because of my job and experience but actual video forensic experts exist now who already specialize in this sort of thing.

I'm saying that I don't have the experience of a video forensics expert but with the limited experience in my field Im pretty good at spotting VFX. I looked one up out of curiosity and found some interesting information on how they investigate video.

here is one at random from Detroit.

http://www.videoforensicexpert.com/video-authentication-analysis-edit-detection-and-tampering/

Quote
We begin all forensic video authentication services and analysis investigations with a preliminary analysis.  Throughout this authentication process, our experts review the video recording in depth through frame by frame analysis. The goal of the preliminary analysis is to determine if any signs of tampering, anomalies or other red flags are present.

I don't think proving a deep fake is any different than normal video analysis. At this point its probably even easier to spot than other methods of fakery which already have a system of investigation in place.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 15, 2019, 01:35:57 PM
I think we already had a case similar to this

https://en.wikipedia.org/wiki/Hustler_Magazine_v._Falwell

Was that the trial where Larry Flynt showed up in court wearing a diaper made out of the American flag and a baby bib with a Nazi medal pinned on?
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 15, 2019, 01:57:10 PM
I think we already had a case similar to this

https://en.wikipedia.org/wiki/Hustler_Magazine_v._Falwell

Was that the trial where Larry Flynt showed up in court wearing a diaper made out of the American flag and a baby bib with a Nazi medal pinned on?

no this was the supreme court case where he allegedly behaved himself.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 15, 2019, 02:04:30 PM
I’d still like to hear what exactly the OP wants to ban.  I suspect he has no real idea himself.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 15, 2019, 02:44:04 PM
I'd like to know who he expects to institute and enforce this ban, along with a description of the investigative and punitive practices he expects those authorities to take.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 15, 2019, 04:40:49 PM
I think we already have the laws in place to stop the criminal aspect of this and he may not in his country.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 15, 2019, 06:05:54 PM
I’d still like to hear what exactly the OP wants to ban.  I suspect he has no real idea himself.

Sorry, my bad. I focused so much on the other points raised that I've forgotten to answer you.

Well, I just think this "ban" would mean a temporary hold on creating and sending money to start new research on this area... Or maybe a summit with scientists from this area to discuss it further in detail before continuing. I think everybody should pause for a year or 2 to calm down, and understand the real impacts and consequences of this. This literally will change forever our perception of reality... its too disruptive to be simply treated as any other study.

You guys can be 100% sure. This technology WILL be weaponized. Imagine people in 2025, scrolling their Facebook timelines, and encountering many videos. One of them is a pixel perfect fake, with the face of a very important scientist saying "Well, it was proven that vaccines do cause autism" or "Climate change is not a thing."

And our perception from the past will change too! We will start to see videos stating that they represent history... but they are completely fake.

Its more than obvious that a very cautious analysis must take place.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 15, 2019, 06:46:52 PM
I don't disagree that this tech is going to be used for some fucked-up purposes. I'm just wondering what can be done about it.

There are no gatekeepers to the development this kind of technology, so it's not like a "summit" of scientists can really accomplish anything. Some developers work on these kinds of projects just out of personal interest. I would imagine that most of the research grants for this kind of thing (if such grants exist) are coming through an intelligence or defense-related agency like DARPA.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 15, 2019, 06:50:32 PM
I’d still like to hear what exactly the OP wants to ban.  I suspect he has no real idea himself.

Sorry, my bad. I focused so much on the other points raised that I've forgotten to answer you.

Well, I just think this "ban" would mean a temporary hold on creating and sending money to start new research on this area... Or maybe a summit with scientists from this area to discuss it further in detail before continuing. I think everybody should pause for a year or 2 to calm down, and understand the real impacts and consequences of this. This literally will change forever our perception of reality... its too disruptive to be simply treated as any other study.

You guys can be 100% sure. This technology WILL be weaponized. Imagine people in 2025, scrolling their Facebook timelines, and encountering many videos. One of them is a pixel perfect fake, with the face of a very important scientist saying "Well, it was proven that vaccines do cause autism" or "Climate change is not a thing."

And our perception from the past will change too! We will start to see videos stating that they represent history... but they are completely fake.

Its more than obvious that a very cautious analysis must take place.

How would you enforce that? The code for some of this software (again the keanu video) is available on github which makes me doubt that any money is being spent on "research" unless they are doing it for a private company making entertainment based software like Adobe or Autodesk.

I also have extreme doubts that a "pixel perfect"  image will ever be available, even in the next 5 years.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 15, 2019, 07:20:13 PM
The Pentagon is already considering this technology as a threat. Also, the 2020 election will be targeted.

https://youtu.be/9OIFVm0dPLw
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 15, 2019, 09:07:56 PM
@Captain Video
This is where the word "Deep" comes from. This is why this is fundamentally different from usual compositing... And the most worrying part? Deepfakes uses Generative Adversarial Networks. Did some genius developed an algorithm that is able to detect fake videos? A person with little knowledge of programming will simply copy that code, and USE IT as the Discriminator. The AI generates an image. Can the Discriminator detect that is fake? Change parameters and try again... After some iterations, the generator will create a frame, and the Discriminator will tell it is real... And so the next frame in the video is created. By design, this algorithm WILL ALWAYS generate fakes that the detection algorithms failed to flag. Also, keep in mind that another layer of AI is used that improves each guess for the frame. This means that for the first frames the Generator will fail to evade the Discriminator in many iterations... but after a while, it will learn how to efficiently avoid the Discriminator with less iterations. Deepfakes, by design, will always be one step ahead....  :-\

(https://skymind.ai/images/wiki/GANs.png)
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on July 15, 2019, 10:04:10 PM
First off: too late. The cat's out of the bag.

Second: who's going to stop Russian and Chinese hackers from accessing the technology illegally? We can't even stop spammers and scammers - this technology is no different.

Third: we've been developing photorealistic CGI for decades now. This is just another step in the same process. A step that uses a new technology, sure, but ray-tracing was a new technology once. No-one's calling for the banning of other CGI technology.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 15, 2019, 10:21:24 PM
So are you saying that you want to ban research on neural networks and genetic algorithms?
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 15, 2019, 10:56:30 PM
You guys are right... there is no stopping it...

Understanding the horrible and destructive ways in which this technology will be used, I just felt that something should be done...

Maybe not even the Pentagon would be able to do anything about it...
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 16, 2019, 01:26:19 AM
I still don't think this is as undetectable as you think.

As far as face swapping or any swapping you have to match camera, lens, lighting, pixel size, artifacting and many other elements not to mention additional compositing.  I don't believe this will be "pixel perfect" or otherwise undetectable in the next 10 years and would be spotted by simple eye examination by a professional who could prove exactly why the video is a fake.

This would also be true of any deep fake regardless of type where the audio was altered. We have specialists who can analyze this already as the ability to manipulate audio has been almost perfect and undetectable for an untrained ear for some time now. People don't get away with it because when you look at the waveform it becomes obvious that the audio is fake or manipulated in some way.

There are some concerns, as you explained its going to be easier to fool your neighbor as well as the average FB user with video and people are going to have to be more skeptical about it. Just like memes and fake news, some people will believe the fakes even when you do prove them incorrect, whats new?

I cant wait for  "Penn and Tellers "Fuck with your friends with deep fakes app"   
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 16, 2019, 05:16:57 AM
Another thing to think about is that the "deep fake" videos typically involve compositing of two or more images. That means that the originals had to come from somewhere.

For example if you're altering a video of a president giving a speech, you'd probably have to start with some source material that is probably already in the public domain, and that video might have some recognizable elements that indicate its location and time.

So even if the illusion is rendered so convincingly that the difference cannot be discerned through casual viewing, certain clues might be detectable that give away the identity of the original source material.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 16, 2019, 06:53:29 AM
Guys, check this out: this is the level of perfection that the AIs can reach when faking voices. I don't know about the audio spectre analysis... but to the human ears, I'd say the technology today already is 95% of the way.

https://youtu.be/DWK_iYBl8cA
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: stands2reason on July 16, 2019, 10:59:26 AM
Second: who's going to stop Russian and Chinese hackers from accessing the technology illegally? We can't even stop spammers and scammers - this technology is no different.

Exactly. If you criminalize deepfakes, only criminals will continue to use deepfakes. We should just voluntary stop technical advancement and let other countries take the tech lead instead...
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 16, 2019, 12:53:28 PM
Another thing to think about is that the "deep fake" videos typically involve compositing of two or more images. That means that the originals had to come from somewhere.

For example if you're altering a video of a president giving a speech, you'd probably have to start with some source material that is probably already in the public domain, and that video might have some recognizable elements that indicate its location and time.

So even if the illusion is rendered so convincingly that the difference cannot be discerned through casual viewing, certain clues might be detectable that give away the identity of the original source material.
\
thats my point with the pixels. The public domain video is most likely under 1080p and already encoded for the internet, likely using an H264 codec or similar. Just using the face pixels reduces this to an even smaller size, then you compost that face onto a different video. No matter how good it looks on the final composite if you zoom in to pixel size there will be a noticeable difference between the two sets of pixels.

garbage in = garbage out.

I'm not saying you cant composite with h264 but it would make it much harder to make it look good and it wont be pixel perfect, regardless of how much time was spent on it.

if you are starting with two prores 444 or uncompressed videos that were shot on the same camera with the same lens and the same lighting maybe but even then you would notice subtle differences in the composite when zooming in.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 16, 2019, 01:07:55 PM
Am I being unreasonable for freaking out because of this? Am I wrong to state that this will forever change our society in a very profound, permanent and negative way?


People are already pointing out the huge danger of that Joe Rogan AI... Someone could get a phone number of someone he knows... Use this AI to say "Let's meet at the street X at 11pm".... and fucking kidnap the person! Or call one of his businesses, act like himself, and then extract confidential information!

I understand that the development of this technology cannot be stopped... So what could we do instead of just sit and wait?
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 16, 2019, 01:20:44 PM
@Captain Video

I think your reasoning is being affected by your Expert bias. You understand all the tools and technologies, and it is extremely easy for you to spot it. I think your bias is preventing you to accurately assess how perfectly real this will look like for the vast majority of the population. You keep bringing up analysis techniques, but that will be completely irrelevant in many, many cases. Think about how many videos you watched in the last month. How many of them did you bother to verify? Like I've said, the answer to the question if it is fake or not can be on the first link of a google search... but the vast majority of the population will not be reached by it.

We must find ways to protect ourselves from the vast damage that this technology will bring... and the solution will not simply be to create detection algorithms. My point is, the detection algorithm can be technically effective... but in reality, SOCIALLY, it will not.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 16, 2019, 01:35:37 PM
@Captain Video

I think your reasoning is being affected by your Expert bias. You understand all the tools and technologies, and it is extremely easy for you to spot it. I think your bias is preventing you to accurately assess how perfectly real this will look like for the vast majority of the population. You keep bringing up analysis techniques, but that will be completely irrelevant in many, many cases. Think about how many videos you watched in the last month. How many of them did you bother to verify? Like I've said, the answer to the question if it is fake or not can be on the first link of a google search... but the vast majority of the population will not be reached by it.

We must find ways to protect ourselves from the vast damage that this technology will bring... and the solution will not simply be to create detection algorithms. My point is, the detection algorithm can be technically effective... but in reality, SOCIALLY, it will not.

I do think you are being unreasonable with your fears

I hear your concern but again this is nothing new.  Just like plenty of people are fooled by Photoshop images they will also be fooled by video, hell people are fooled by video now without the deep fakes.

You often don't need to be an expert when it comes to being skeptical about something.

Someone plays you a video of your wife doing the neighbor, "well that cant be right, perhaps I should check into this before making any rash decisions."  Wife: "thats not me its faked,  I was at work"  no need to analyze the video when you can confirm with a phone call and call her boss (you should believe your wife in the first place)

The more it happens the less people will be fooled by it.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 16, 2019, 01:48:46 PM
BTW,  No one has ever actually called me an expert before, especially in this place, you have done this twice now.  It kind of feels really good.

Seriously, thank you for that.  And thank you for treating this as a conversation and not a war.  I absolutely do see and understand many of your concerns. I agree this is not something to be completely ignored. Skeptics will have to stay on their toes.


Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 16, 2019, 02:35:15 PM
When I am considering whether it is reasonable to ‘freak out’ about something, I always consider whether there is anything I can do about it. If there is, then there is no reason to ‘freak out’ about it; i should just do what I can to fix it. If there is not, then there is no point in ‘freaking out’ about it, since doing so will not improve the situation.  In either case, freaking out is the wrong approach.  And so is calling for the government to ban basic computer science research out of fear.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 16, 2019, 07:30:00 PM
When I am considering whether it is reasonable to ‘freak out’ about something, I always consider whether there is anything I can do about it. If there is, then there is no reason to ‘freak out’ about it; i should just do what I can to fix it. If there is not, then there is no point in ‘freaking out’ about it, since doing so will not improve the situation.  In either case, freaking out is the wrong approach.  And so is calling for the government to ban basic computer science research out of fear.

Well, I might have exaggerated my choice of words a bit. I used the expression "freaking out" only to explicit my levels of concern against the perceived calmness that I've sensed here.
You've spoken like a Stoic. Wise words indeed. I've finished Marcus Aurelius Meditations and I'm starting Epictetus letters now. I'm already habituating myself to live by that attitude.
As for the call for the ban on this research.... Yeah, after starting this topic it quickly become evident to me that that is not feasible.

BTW,  No one has ever actually called me an expert before, especially in this place, you have done this twice now.  It kind of feels really good.
Seriously, thank you for that.  And thank you for treating this as a conversation and not a war.  I absolutely do see and understand many of your concerns. I agree this is not something to be completely ignored. Skeptics will have to stay on their toes.

Well, I'm so glad that this discussion is heading this way... when there is an absolute separation between our arguments and ourselves, is when all of us learn and adapt faster. :D
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 16, 2019, 07:42:18 PM
Am I being unreasonable for freaking out because of this?

No, you are not. This is legitimately freaky. But as The Latinist suggested, holding your emotions in check is a major component of rationality. 

'Deep fakes' are an emerging, revolutionary technology that has the potential to create convincing false narratives by literally putting words into the mouths of prominent leaders. As such, it has the potential to undermine our perceptions of reality in ways we cannot even imagine at this point. There's no telling the level of damage it could cause.


Am I wrong to state that this will forever change our society in a very profound, permanent and negative way?

That remains to be seen, but this is just another step in a destabilization process that is already well underway. For the past couple decades we've been dealing with an erosion of confidence in facts and reality, to the point where our trust in information sources has become practically arbitrary. In this environment, the side of honesty, transparency and science is never going to win out by merely pitting narrative against narrative.

What we need to do is create a new culture that values facts over bullshit. We can start by educating people on the importance of approaching all information in a skeptical way, and empowering everybody with the tools of critical thinking and the techniques for identifying deception.

For one thing, we need to start paying attention to information instead of investing so much confidence in personalities.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on July 16, 2019, 08:50:28 PM
Am I being unreasonable for freaking out because of this?

Yes.

Am I wrong to state that this will forever change our society in a very profound, permanent and negative way?

Yes.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 16, 2019, 09:32:47 PM
I will demonstrate objectively why this technology will be disruptive and damaging to our civilization. Let's begin with a simplification: An Hierarchy of Evidence.

Spoken Word < Written Text < Photography < Audio < Video < Physical Evidence

(By spoken Word I mean someone saying something in real life to you). All these levels are vulnerable to fraud and lies. But these levels of evidence vary in two very important aspects: The difficulty in which they can be distorted, and the perceived value people give to them. Try to imagine, what is the proportion of the human population that is capable to tell a lie? I'd say virtually 100%. Only individuals incapable of speech cannot do it, although they can still lie in other ways. As soon as toddlers begin learning the language, they can start telling lies.
Next level: Written Text. Only 781 million adults are estimated to be illiterate... so around 5.5 Billion people are capable of writing lies.

Now, the next step is when things change radically. Of all the human population, how many of us are really capable of creating realistic adulterated photos? How many people dominate Photoshop in the world? 100 million maybe? What about audio? That one is trickier. We can fraud audio by impersonation too, and not exclusively through technology. But lets say for the sake of the argument that that number is 70 million people.... What about video? How many people in the world dominate Premiere Pro, Vegas, or any other video software? 40 million? I know the numbers are probably wrong, but lets focus on the general argument here. Physical Evidence can be falsified too, as an example of a corrupt cop that plants drugs in someone to falsely accuse them. But in terms of all the information we consume, the vast majority of what we can physically see is extremely rare to be false.

This is the hierarchy as we perceive today, along with the approximate number of people capable of faking it. Red means low confidence in this type of evidence, orange average, green high. Curly Braces represents the realm of the Internet and all media we get information from.

{{{Spoken Word [7.4 Billion] < Written Text [ 5.5 Billion] < Photography [ 0.1 Billion] < Audio [ 0.07 Billion] < Video [0.04 Billion] }}} < Physical Evidence

Keep in mind that what validates everything we read on the internet is located in the expertise areas of Photography, Audio and Video.

Why, contrary to many arguments, this technology is absolutely different from any other forms of frauds? Why deepfakes are so fundamentally more destructive than any other means such as Photoshop, Premiere Pro? After all, even Hitler and Stalin already were doctoring photographs... The whole difference lies in two aspects, impossible prior to the development of this technology. Aspects only possible by the employment of AI.

ABSOLUTE INDEPENDENCE from the fraudster technical skills. This would mean that eventually, people with zero technological skills would be able to create realistic fakes. Want to create a realistic compostion? Work hours with a team to create one. Want to create a deepfake? Use the template and just use one single click.
MASSIVE SCALABILITY This is the one that is going to be disruptive. We are not going to experience this in an incremental way. For example, how many people are capable of making an almost perfect impersonation of Joe Rogan in the United States? Maybe around 2 thousand people? After this technology, how many will be able to? In Gary Oldmans voice: EVERYONE. Thats the thing. As soon as that app is published, the number of perfect Joe Rogan impersonators will jump from 2 thousand to many billions. And the same AI that can perfectly impersonate Joe Rogan... can impersonate you. Your mother. Your accountant. Anyone you trust.... and that is fucking scary.

So, in a matter of A FEW YEARS, civilization's hierarchy of evidence will instantaneously change to something like this:

{{{Spoken Word [7.4 Billion] < Written Text [ 5.5 Billion] < Photography [ 5.5 Billion] < Audio [ 5.5 Billion] < Video [5.5 Billion] }}} < Physical Evidence

This is why things will be much different. The distribution of the content on the internet will sharply move towards the higher levels of Simulacra and Simulation. Our relationship to Photos, Audio and Video will forever change. And this will be terrible for our perception of reality, since the vast majority of our information diet comes from those forms.

https://www.youtube.com/watch?v=Hl9zO0cS-NU

Simulacra and Simulation Stages:

[1] It is the reflection of a basic reality
[2] It masks and perverts a basic reality
[3] It masks the absence of a basic reality
[4] It bears no relation to any reality whatsoever: it is its own pure simulacrum.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on July 16, 2019, 11:39:23 PM
The time to react like this, if there ever was one, which there wasn't, would have been 25 years ago when CGI started becoming more photorealistic.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 17, 2019, 12:04:47 AM
My sister would say, “If it’s not time to do something about it, it’s not time to worry about it.”
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 17, 2019, 04:13:00 AM
We may not be able to stop the technology from being developed, but there certainly are things we can do to mitigate its negative impact.

A recent episode (https://www.stitcher.com/podcast/how-stuff-works/sleepwalkers/e/61437892) of the excellent podcast Sleepwalkers was dedicated to "deep fakes" technology and its potential uses for good and bad.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 17, 2019, 09:55:17 AM
The time to react like this, if there ever was one, which there wasn't, would have been 25 years ago when CGI started becoming more photorealistic.

You are completely missing the two crucial points I've explained. The costs of time and expertise required by traditional CGI are multiple orders of magnitude behind. I'm not exaggerating... in terms of potential output to the internet, we are literally talking about an potential increase in faked content in the orders of 100 MILLION times more than any previous CGI, Photoshop or any other non AI technology.

I will provide a more concrete example to you. Imagine the number of people you encounter every month... at work, university, your family, your friends, at the shopping malls... If I asked you to find to me how many of them are actually capable of mixing in a photo-realistic way two faces in Photoshop, to the point of quasi-perfection... how many of them would be able to? 1% maybe? And how many hours will they spend on crafting this? Now... With this technology, how many of them would be able to achieve the same task? Again, in Garry Oldmans voice: EVERYONE. You could literally gather a team with 100 Photoshop experts and given the task of creating the maximum number of fake realistic faces as possible... at the same time, give this new technology to a 3 year old toddler and tell him/her to keep pressing the "Generate X Faces button". The toddler would be able to output realistic fake content literally more than a thousand times more than the team of Photoshop experts.

(https://d2r55xnwy6nx47.cloudfront.net/uploads/2019/03/FacesGrid_560.jpg)

More people that don't exist, created by AIs.

(https://cdn.vox-cdn.com/thumbor/CMJs1AJyAmf27RUd2UI5WBSZpy4=/0x0:3049x2048/920x613/filters:focal(1333x1562:1819x2048):format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/63058104/fake_ai_faces.0.png)

That's what I am talking about. Oh... Someone mentioned Russia or China on this thread. People are pointing out that an army of bots on twitter and Facebook is already being formed by these fake people. This is just another example of the very destructive, Orwellian powers of this technology, that were completely impossible before AI. What would be the costs for a government to pay professionals to Photoshop 20 million realistic faces? And what is the cost to do the same thing with AI? Like I've said, it's a drop of money and computational resources in the order of more than 100,000 times... This is not incremental. This is a huge step. Hugely disruptive.

For more instantenously GAN AI faces of people that dont exist, go to this website and keep refreshing the page.

https://www.thispersondoesnotexist.com/
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Gigabyte on July 17, 2019, 10:09:49 AM
Second: who's going to stop Russian and Chinese hackers from accessing the technology illegally? We can't even stop spammers and scammers - this technology is no different.
Society is not ready for constant spam calls.  The technology used must be terminated.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 17, 2019, 10:21:52 AM
Simulacra and Simulation Stages:

[1] It is the reflection of a basic reality
[2] It masks and perverts a basic reality
[3] It masks the absence of a basic reality
[4] It bears no relation to any reality whatsoever: it is its own pure simulacrum.

Today in our civilization, less than 3% of people are capable of realistically creating Stage 2 Simulacra. After this technology, over just one or two years, 4 Billion people will be able to create Stage 3 Simulacra... and that will permanently weaken our perception of reality by many orders of magnitude.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 17, 2019, 11:50:02 AM
Did you miss the memo? We're already in a post-truth world.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 17, 2019, 02:34:50 PM
This new technology just adds videos of people talking to the list of media types that we can no longer trust at face value.

The solution is to learn how to properly vet information before believing it or forwarding it on to others.

What else can we do? Maybe we could develop some kind of machine learning tech that inputs news articles, automatically tracks down sources, compares those sources against a database of known news agencies, and returns some kind of verification score.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 17, 2019, 02:36:12 PM
The Benefits of AI Employees: They can work tirelessly 24 hours a day, No difficult contract negotiations to go through, no vacations, no sick leaves... One single human is going to be capable of controlling a whole AI "Staff" in a Studio...

https://www.youtube.com/watch?v=bmqd9nYH5Fw

 
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: stands2reason on July 17, 2019, 04:00:44 PM
The Benefits of AI Employees: They can work tirelessly 24 hours a day, No difficult contract negotiations to go through, no vacations, no sick leaves... One single human is going to be capable of controlling a whole AI "Staff" in a Studio...

https://www.youtube.com/watch?v=bmqd9nYH5Fw

Deus Ex already did it...

https://deusex.fandom.com/wiki/Eliza_Cassan
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 17, 2019, 04:40:21 PM
The Benefits of AI Employees: They can work tirelessly 24 hours a day, No difficult contract negotiations to go through, no vacations, no sick leaves... One single human is going to be capable of controlling a whole AI "Staff" in a Studio...

https://www.youtube.com/watch?v=bmqd9nYH5Fw

What a brilliant way to make TV news even more boring.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 17, 2019, 04:49:34 PM
(click to show/hide)

Deus Ex already did it...

https://deusex.fandom.com/wiki/Eliza_Cassan

Naming that character "Eliza Cassan" was a brilliant bit of wordplay, in that it obliquely references the 1940s-'50s Hollywood film director Elia Kazan (https://en.wikipedia.org/wiki/Elia_Kazan). Kazan was known for pioneering a style of "cinematic realism," which incorporated real-world social and political themes into movies which appeared very realistic and down-to-earth.

In the game, that old Hollywood reference is juxtaposed against a "neo-noir" cyberpunk environment wherein everything appears super highly stylized. Kinda cool.

I love it when pop-culture games and movies employ clever allusions like that.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 17, 2019, 05:31:59 PM
@John Albert
That's really interesting! I've played a lot of Deus Ex and didn't know that. By the way... I have a poster with 33 signatures from the devs... including Adam Jensen himself!

Reading the article about this AI news anchor from Deus Ex just got seriously creepy and ominous.

Description from a Dystopic, fucked up world of Deus Ex:

"There, he discovers that the famous celebrity newsreader is actually an advanced artificial intelligence, created and designed for the purpose of controlling public information and shaping mass discourse, with the eventual goal of control and manipulation of all available information. The extremely advanced AI is programmed to guide public opinion to certain ends and was created by the conspiracy for this purpose. "

Real life: Chinese government spending millions on creating an AI to present the news...
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 17, 2019, 06:09:50 PM
Holy shit... I've just realized that a true conspiracy might be taking place right now... AIs need training. AIs need a lot of data. How could an ill intentioned agency, government or business improve its algorithm? By creating means in which a massive number of people help to train the AI... the deployment of an entertainment app would be perfect for this.... Tens of millions of people sending their photos, and training the algorithm by giving extremely important feedback that the AI cannot get by itself... If the person generates a deepfake and instantly delete it, the app learns that it was probably bad in some aspect. The person liked it and shared on facebook? The app learns that that fake was real enough, and reinforces the correspondent neurons from the network. Instead of relying on the feedback of a few employees... this is a way to get tens of millions of people working for free,  24 hours a day, in improving and providing feedback to the AI.

A dystopic conspiracy that not only survives the Occams Razor principle ( contrary to all the others ), but also literally is the most effective and simple way,  computationally speaking. Also, from the business point of view, makes the most sense.

People are downloading and interacting with deep fake apps for fun... unknowing that they are actively training the network, for whatever purpose the owners might have for the AI in the future.

EDIT: Well... I've quickly realized that what i wrote above is something extremely obvious, and apparently there are a lot of known cases already... the most recent one:

Millions of people are training the AI... of Russia.

Millions of Americans... training and providing all the personal data on their phones to an AI... that it is going to be used to create an army of bots to manipulate them... in exchange for MEMES. The real world just combined the vigilance of 1984 with the mindless pleasure-seeking of Brave new world.

FaceApp's viral success proves we will never take our digital privacy seriously
https://edition.cnn.com/2019/07/17/tech/faceapp-privacy-concerns/index.html
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on July 17, 2019, 06:50:07 PM
Igor, I suspect a few of us have been thinking about this stuff for quite some time. In my case, this kind of simulation is natural and expected progression of technologies like ELIZA from many decades ago (https://en.wikipedia.org/wiki/ELIZA) - half a decade before I was born - so I've been expecting AI and artificial humans for my entire life. The mass manipulation of natural born humans by artificial human representations isn't something that was predicted much, I think. It doesn't surprise me at all, however, and it also doesn't particularly frighten me. Humans have always played a game of technological development with offensive and defensive advances driving us to ever greater power.

Right this minute there are AI agents that effortlessly identify deepfakes and other manipulated video. I expect Google and Facebook to integrate them into their algorithms in various ways before 2020 is out. This is a new kind of spam, and it needs a new kind of spam filter. That already exists in a nascent form, just like the spam exists in a nascent form.

This is a serious problem with serious consequences, and in need of serious solutions, but I think we're going to be fine (from this particular dystopian horror at least).
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 17, 2019, 07:42:04 PM
I understand. But my point is, this is going to be disruptive.  And the general population must be critical and skeptical about it. I'm trying to point out the absurd chasm that exists around the perceived uses of this technology.

General population: LOL look how freakin awesome this deepfake is! // OMG Keanu Reeves! // Oh... did you already uploaded your photo to that app that shows how you are going to look like when you are older? Post the link on the company's group so everybody can download it!

What governments are doing:
In the United States the Pentagon and DARPA are already spending millions to defend itself against the weaponization of deepfakes.

Russians ( and probably every other government ) are creating an army of deepfake bots on Facebook and Twitter to disrupt elections to their advantage.

The Chinese Government is so far ahead, that they are actually,  LITERALLY, implementing a Big Brother AI. Let that sink in. An AI that is capable to be aware of all citizens, where they go, what they do, and who they associate with. Not only that, the AI rewards you for accusing your neighbors that are against the government. An AI that punishes you if you "associate" with the wrong people. An AI that can listen and process in real time,  millions of microphones in people's mobiles.  An omniscient AI that rewards and punishes you according to the government desires. This is not a far distant future... 1984 in fact, officially begins in 2020.

https://youtu.be/eViswN602_k
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: stands2reason on July 17, 2019, 07:48:44 PM
If you want a picture of the future, imagine a boot stamping on a Deepfake face—forever.

https://www.youtube.com/watch?v=dX1f6H78Y_k
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 17, 2019, 08:48:28 PM
Elon Musk announces NeuraLink progress... It will allow AIs to receive and input data directly into our brains. In objective terms, the computer-brain interface just gave a leap of 1 thousand times better than previous technology.

https://youtu.be/lA77zsJ31nA
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on July 17, 2019, 09:26:31 PM
Anyone remember Ananova (https://en.wikipedia.org/wiki/Ananova)?
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on July 17, 2019, 09:30:46 PM
The time to react like this, if there ever was one, which there wasn't, would have been 25 years ago when CGI started becoming more photorealistic.

You are completely missing the two crucial points I've explained. The costs of time and expertise required by traditional CGI are multiple orders of magnitude behind...

No, I'm not. This is not a new technology. It's the same technology, just better. Like all technology, we improve it over time. The technology used in Tron is orders of magnitude behind the technology used in Thor: Ragnarok but it's still essentially the same technology, that we've been developing and improving for decades. This is just another step in the same direction.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 17, 2019, 10:15:50 PM
About the Russians:

What remains concerning, however, is the language in the app's terms of service. In one densely-worded section, the company informs users that they "grant FaceApp a perpetual, irrevocable, nonexclusive, royalty-free, worldwide, fully-paid, transferable sub-licensable license to use, reproduce, modify, adapt, publish, translate, create derivative works from, distribute, publicly perform and display your User Content and any name, username or likeness provided in connection with your User Content in all media formats and channels now known or later developed, without compensation to you."

Millions of Americans are TRAINING and PROVIDING PERSONAL DATA to Russia's AI... For free!

Viral App FaceApp Now Owns Access To More Than 150 Million People's Faces And Names
https://www.forbes.com/sites/johnkoetsier/2019/07/17/viral-app-faceapp-now-owns-access-to-more-than-150-million-peoples-faces-and-names

"You might end up on a billboard somewhere in Moscow, but your face will most likely end up training some AI facial-recognition algorithm."
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: stands2reason on July 18, 2019, 10:27:03 AM
(https://imgs.xkcd.com/comics/predictive_models.png)
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: SnarlPatrick on July 19, 2019, 09:20:18 AM
You're right to be concerned. I think many of the examples you gave are plausible. But there's no stopping it. We've never been able to "contain" technology before and there's no reason to think we will be able to this time. We will have our first court cases test this technology and soon we will have precedents. Hopefully, it will be illegal to maliciously depict others doing something they have not. But it will be comedically omnipresent. Outlawing it all-together seems like a fool's errand when you'll be able to do it from your laptop.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 19, 2019, 12:26:18 PM
Hopefully, it will be illegal to maliciously depict others doing something they have not.

That's what I am talking about... We are dealing with new, never before seen cases that enter into the realm of the grey zone of the law.

Take FaceApp for example. A Lot of celebrities used it... And by the user agreement, the company now is entitled by law to do whatever it wants with their image... Think about how profound and serious this is. Read the user agreement, thinking that you are a celebrity, and that the app reportedly had access to all the camera roll on their phones.

"grant FaceApp a perpetual, irrevocable, nonexclusive, royalty-free, worldwide, fully-paid, transferable sub-licensable license to use, reproduce, modify, adapt, publish, translate, create derivative works from, distribute, publicly perform and display your User Content and any name, username or likeness provided in connection with your User Content in all media formats and channels now known or later developed, without compensation to you."

By law... now they are allowed to create deepfakes with their faces. They are allowed to use their image on ads. They are allowed to use all the images to feed a database and train an AI for whatever purposes they want.

This is my greatest concern. Since it is impossible to stop the technology... how can we structure our societies to prevent further damage? The developments in the AI field are getting so fast, that legislation will constantly lag behind in terms of protecting us from their malicious uses...
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on July 19, 2019, 01:58:34 PM
I suspect that the TOS will be ruled to be an illegal contract, and voided.

ETA: Not to avoid your larger point, which is an ongoing challenge.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: bimble on July 19, 2019, 02:27:29 PM
I seem to recall that Twitter has the same bit in their T&C...
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 19, 2019, 02:28:16 PM
I suspect that the TOS will be ruled to be an illegal contract, and voided.

ETA: Not to avoid your larger point, which is an ongoing challenge.

On what grounds? US courts have routinely upheld clickwrap agreements like that.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 19, 2019, 03:42:01 PM
"Look all the faces and names that we now have all the legal rights to do whatever we want!"

(https://img-9gag-fun.9cache.com/photo/aA3WOKZ_700bwp.webp)
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 19, 2019, 09:15:24 PM
I think there might be a huge loophole on those Laws... The end user agreement says that they will be entitled to do whatever they want with the photo you send... But what if you send the picture of a famous person (As many people did)? Or someone else? Will they still have the right?
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 20, 2019, 12:21:53 AM
There you have it... AIs learning how to temper CCTV footage.

https://9gag.com/gag/aQRwVYw
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on July 20, 2019, 04:35:04 PM
I suspect that the TOS will be ruled to be an illegal contract, and voided.

ETA: Not to avoid your larger point, which is an ongoing challenge.

On what grounds? US courts have routinely upheld clickwrap agreements like that.

It looks tp my inexpert eye like the TOS May be too broad. I can't give you the rights to use George Clooney's image that way even if I have a picture I took of him on my phone.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on July 23, 2019, 12:00:36 AM
An article.

To fix the problem of deepfakes we must treat the cause, not the symptoms (https://www.theguardian.com/commentisfree/2019/jul/23/to-fix-the-problem-of-deepfakes-we-must-treat-the-cause-not-the-symptoms)

Quote
We haven’t yet seen a clear frontrunner emerge as the Democratic candidate for the 2020 US election. But I’ve been interested in another race – the race to see which buzzword is going to be a pivotal issue in political reporting, hot takes and the general political introspection that elections bring. In 2016 it was “fake news”. “Deepfake” is shoring up as one of the leading candidates for 2020.

This week the US House of Representatives intelligence committee asked Facebook, Twitter and Google what they were planning to do to combat deepfakes in the 2020 election. And it’s a fair question. With a bit of work, deepfakes could be convincing and misleading enough to make fake news look like child’s play.

Deepfake, a portmanteau of “deep learning” and “fake”, refers to AI software that can superimpose a digital composite face on to an existing video (and sometimes audio) of a person.

The term first rose to prominence when Motherboard reported on a Reddit user who was using AI to superimpose the faces of film stars on to existing porn videos, creating (with varying degrees of realness) porn starring Emma Watson, Gal Gadot, Scarlett Johansson and an array of other female celebrities.

However, there are also a range of political possibilities. Filmmaker Jordan Peele highlighted some of the harmful potential in an eerie video produced with Buzzfeed, in which he literally puts his words in Barack Obama’s mouth. Satisfying or not, hearing Obama call US president Trump a “total and complete dipshit” is concerning, given he never said it.

Click through for lots more words. The website's The Guardian, which is a reputable news site.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: 2397 on July 23, 2019, 05:01:23 AM
I think there might be a huge loophole on those Laws... The end user agreement says that they will be entitled to do whatever they want with the photo you send... But what if you send the picture of a famous person (As many people did)? Or someone else? Will they still have the right?

That is one of the main problems with sites like Facebook, people sharing private images and information about people other than themselves without asking them, people who might not even be on Facebook.

If you're not paying then you're the product, okay, if the terms have been explicit. But if you're not even participating, and you're still the product? If we can't find some way to stop the abuse of data, I'd rather have a total ban on the commercial use of and commercial entities sharing user data. There are other ways to pay for content and hosting, and you can still have ads without knowing who you're advertising to.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: SnarlPatrick on July 26, 2019, 01:22:29 PM
Targetted adverising is too big an industry to vanish. It'll never happen.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Shibboleth on July 26, 2019, 04:50:12 PM
I remember when the ICS used deepfake technology to make it look like Ben Richards died in the game zone to Captain Freedom. I lost a lot of money due to that fake. I endorsed Ben Richards. That boy is one bad motherfucker.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 27, 2019, 01:54:17 PM
https://youtu.be/xlrGOfvYcQc
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on July 29, 2019, 11:23:01 AM
On the positive side of the ledger:

(https://i.imgur.com/kgJwwQU.mp4)
(https://imgur.com/gallery/Npd8Vzd)
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on July 30, 2019, 11:37:01 AM
By  :steve:  himself:
The Challenge of Deepfakes
https://theness.com/neurologicablog/index.php/the-challenge-of-deepfakes/
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 30, 2019, 11:43:07 AM
I dont fully agree with Steve either.

This is fear and I think people should relax, this same fear has come up many times before and we get through it every time.

The name makes people even more afraid, we should stop using it.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: stands2reason on July 30, 2019, 12:16:27 PM
I want to see CGI movies. CGI movies of old actors, synthetic actors, celebrity parodies, all of it. Obama cameo? It better happen at some point.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Captain Video on July 30, 2019, 01:07:26 PM
I want to see CGI movies. CGI movies of old actors, synthetic actors, celebrity parodies, all of it. Obama cameo? It better happen at some point.

Your comment made me wonder what SAG-AFTRA will have to say about "old actors, synthetic actors, celebrity parodies" and I found these articles.

https://www.reuters.com/article/us-film-resurrections-analysis/actors-seek-posthumous-protections-after-big-screen-resurrections-idUSKBN14J1TU

Quote
Filmmakers are tapping advances in digital technology to resurrect characters after a performer dies, most notably in “Rogue One: A Star Wars Story.” The film, in theaters now, features the return of Grand Moff Tarkin, originally played by a long-dead actor.

The trend has sent Hollywood actors in the here-and-now scrambling to exert control over how their characters and images are portrayed in the hereafter.

“Celebrities are increasingly involved in making plans to protect their intellectual property rights,” said Mark Roesler, an attorney and chairman of CMG Worldwide, an agency representing celebrity estates. “They understand that their legacy will continue beyond their lifetime.”


https://deadline.com/2018/04/deepfaking-technology-sag-aftra-actors-union-fighting-back-1202371117/

Quote
“SAG-AFTRA has undertaken an exhaustive review of our collective bargaining options and legislative options to combat any and all uses of digital re-creations, not limited to deepfakes, that defame our members and inhibit their ability to protect their images, voices and performances from misappropriation. We are talking with our members’ representatives, union allies, and with state and federal legislators about this issue right now and have legislation pending in New York and Louisiana that would address this directly in certain circumstances. We also are analyzing state laws in other jurisdictions, including California, to make sure protections are in place. To the degree that there are not sufficient protections in place, we will work to fix that.


I think its going to be hard to pull off fakes without using copyrighted materials and SAG-AFTRA seems to be all over this already. If its properly labeled as a parody they probably wont be able to do anything but if non creative commons source materials were used in the making of the video there will be a legal right to stop it. I imagine this will end up much like music sampling.

Nothing about fully synthetic CGI actors but I think that's a bit further away tech wise.

Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: stands2reason on July 30, 2019, 01:47:18 PM
Nothing about fully synthetic CGI actors but I think that's a bit further away tech wise.

But could you seed it with actual celebrity likeness, and then add some variation to make it just distinct enough?
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Shibboleth on July 30, 2019, 04:10:34 PM
I am guessing that in the future deep fake technology will be used in films to make people's faces more symmetrical and to remove blemishes, etc. Basically real time airbrushing.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on July 30, 2019, 10:23:17 PM
Shibboleth, that's a fascinating idea. I bet it could be used to morph eye position - as Apple is starting to do - so we actually look like we're looking at the camera in a video call.

The Stuff You Should Know guys did an ep on this topic this week. I'd put them in the 3/4 range for oveall accuracy - normal for them, and significantly higher than most shows where I know what's what.

I enjoy their banter - which is a significant portion of the show. They're one of the oldest podcasts out there, so they're doing something appealing. :)

https://www.stuffyoushouldknow.com/podcasts/will-deepfakes-ruin-the-world.htm (https://www.stuffyoushouldknow.com/podcasts/will-deepfakes-ruin-the-world.htm)
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on July 30, 2019, 10:29:05 PM
I've tried listening to a few episodes, but they always sound like they have no idea what they're talking about.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 31, 2019, 01:40:34 AM
I also tried listening to that podcast, but found their explanations of things I do know so muddled and often plain wrong that I didn’t trust anything they said that I didn’t already know.  Kinda made the whole exercise pointless.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on July 31, 2019, 08:22:15 AM
I listened to a few episodes of Stuff You Should Know about 10 years ago, but had to quit in disgust at their fear-mongering about fluoride, and enthusiastic endorsement of alt-med practices like chiropractic and colonic irrigation.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on July 31, 2019, 08:48:56 AM
Yeah, I'm not endorsing their accuracy in general. I like their banter and use the topics as a starting point for further learning, generally. In this case a layperson with no idea what deepfakes are would come away with a reasonably accurate orientation to the topic.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on July 31, 2019, 09:38:13 PM
Yeah, I'm not endorsing their accuracy in general. I like their banter and use the topics as a starting point for further learning, generally. In this case a layperson with no idea what deepfakes are would come away with a reasonably accurate orientation to the topic.

But, because of the source, would have no way of knowing whether that ‘orientation’ was accurate or utter bullshit, which makes the information worthless.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on July 31, 2019, 11:50:31 PM
Yeah, I'm not endorsing their accuracy in general. I like their banter and use the topics as a starting point for further learning, generally. In this case a layperson with no idea what deepfakes are would come away with a reasonably accurate orientation to the topic.

But, because of the source, would have no way of knowing whether that ‘orientation’ was accurate or utter bullshit, which makes the information worthless.

In this specific case I am saying that based on my knowledge this episode is accurate enough to be worthwhile for a layperson. The hosts are personable and engaging, which helps with the 'layperson listening' part.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on August 01, 2019, 01:56:22 AM
Yeah, I'm not endorsing their accuracy in general. I like their banter and use the topics as a starting point for further learning, generally. In this case a layperson with no idea what deepfakes are would come away with a reasonably accurate orientation to the topic.

But, because of the source, would have no way of knowing whether that ‘orientation’ was accurate or utter bullshit, which makes the information worthless.

In this specific case I am saying that based on my knowledge this episode is accurate enough to be worthwhile for a layperson. The hosts are personable and engaging, which helps with the 'layperson listening' part.

My point is that, however personable the hosts, information they give is useless to a ‘layperson listening’ if he cannot know it’s value without already knowing it.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on August 01, 2019, 01:59:06 AM
Yeah, I'm not endorsing their accuracy in general. I like their banter and use the topics as a starting point for further learning, generally. In this case a layperson with no idea what deepfakes are would come away with a reasonably accurate orientation to the topic.

But, because of the source, would have no way of knowing whether that ‘orientation’ was accurate or utter bullshit, which makes the information worthless.

In this specific case I am saying that based on my knowledge this episode is accurate enough to be worthwhile for a layperson. The hosts are personable and engaging, which helps with the 'layperson listening' part.

My point is that information is useless to a ‘layperson listening’ if, because of the poor quality of the source, he cannot know the value of any information he does not already know.

If the layperson listening had never listened to the podcast before, they wouldn't have any basis to determine the value of the information. It would just be information.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on August 01, 2019, 02:05:12 AM
Yeah, I'm not endorsing their accuracy in general. I like their banter and use the topics as a starting point for further learning, generally. In this case a layperson with no idea what deepfakes are would come away with a reasonably accurate orientation to the topic.

But, because of the source, would have no way of knowing whether that ‘orientation’ was accurate or utter bullshit, which makes the information worthless.

In this specific case I am saying that based on my knowledge this episode is accurate enough to be worthwhile for a layperson. The hosts are personable and engaging, which helps with the 'layperson listening' part.

My point is that information is useless to a ‘layperson listening’ if, because of the poor quality of the source, he cannot know the value of any information he does not already know.

If the layperson listening had never listened to the podcast before, they wouldn't have any basis to determine the value of the information. It would just be information.

Information on which they should not rely because the people giving it are bullshit artists. This is like quoting Donald Trump as a source because he happens to have said something that’s not a lie once. Whether the thing he said is true or not, his words are useless because he’s a liar.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on August 01, 2019, 02:11:13 AM
Yeah, I'm not endorsing their accuracy in general. I like their banter and use the topics as a starting point for further learning, generally. In this case a layperson with no idea what deepfakes are would come away with a reasonably accurate orientation to the topic.

But, because of the source, would have no way of knowing whether that ‘orientation’ was accurate or utter bullshit, which makes the information worthless.

In this specific case I am saying that based on my knowledge this episode is accurate enough to be worthwhile for a layperson. The hosts are personable and engaging, which helps with the 'layperson listening' part.

My point is that information is useless to a ‘layperson listening’ if, because of the poor quality of the source, he cannot know the value of any information he does not already know.

If the layperson listening had never listened to the podcast before, they wouldn't have any basis to determine the value of the information. It would just be information.

Information on which they should not rely because the people giving it are bullshit artists. This is like quoting Donald Trump as a source because he happens to have said something that’s not a lie once. Whether the thing he said is true or not, his words are useless because he’s a liar.

But without having listened to the show before, they have no way to know that.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on August 01, 2019, 09:08:37 AM
Yeah, I'm not endorsing their accuracy in general. I like their banter and use the topics as a starting point for further learning, generally. In this case a layperson with no idea what deepfakes are would come away with a reasonably accurate orientation to the topic.

But, because of the source, would have no way of knowing whether that ‘orientation’ was accurate or utter bullshit, which makes the information worthless.

In this specific case I am saying that based on my knowledge this episode is accurate enough to be worthwhile for a layperson. The hosts are personable and engaging, which helps with the 'layperson listening' part.

My point is that information is useless to a ‘layperson listening’ if, because of the poor quality of the source, he cannot know the value of any information he does not already know.

If the layperson listening had never listened to the podcast before, they wouldn't have any basis to determine the value of the information. It would just be information.

Information on which they should not rely because the people giving it are bullshit artists. This is like quoting Donald Trump as a source because he happens to have said something that’s not a lie once. Whether the thing he said is true or not, his words are useless because he’s a liar.

I don't understand your apparant hostility here. I agree that an unreliable source should not be trusted blindly.

*This episode* has information that is *accurate enough* for a *basic orientation* to the topic.

I would be comfortable suggesting a non-podcast-listening friend give this single episode a listen within the constraints noted above. That judgement is based on my expertise (limited as it is) and my assessment of their accuracy.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on August 01, 2019, 08:06:07 PM
I get the impression that their level of accuracy is highly variable, not that everything that they say is bullshit. They are unreliable, but you can't prejudge any single subject they talk about.

For me it was their delivery that put me off, not the quality of the information.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on August 01, 2019, 08:55:55 PM
The Shining starring Jim Carrey : Episode 1 - Concentration [DeepFake]
https://www.youtube.com/watch?v=HG_NZpkttXE
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Soldier of FORTRAN on August 07, 2019, 12:53:01 AM
Reference Image for D&D: Doppleganger face transitions (starts ~0:53)
https://www.youtube.com/watch?v=VWrhRBb-1Ig


This one's just funny
https://www.youtube.com/watch?v=jrG-REWdV6A


Bonus: Mage who spends the first few rounds just buffing
(click to show/hide)

Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on August 22, 2019, 04:51:58 PM
Jordan Peterson wrote a text about literally the very problem that I raised on this thread. People are already using his voice to make him say whatever they want... And that already is extremely harmful and stressful to him. And remember... The AI that can learn how to impersonate Jordan Peterson, is the same that can learn YOUR VOICE.

I Didn’t Say That
https://www.jordanbpeterson.com/blog-posts/i-didnt-say-that/

Quote
(...) This week, however, a company called notjordanpeterson.com put an AI engine online that allows anyone to type anything and have it reproduced in my voice. It’s hard to get access to or use the site, at the moment, presumably because it is currently attracting more traffic than its servers can handle. A variety of sites that pass themselves off as news portals—and sometimes are—have either reported this story straight (Sputnik News) or had a field day (Gizmodo) having me read, for example, the SCUM manifesto (hypothetically an acronym for Society for Cutting Up Men), a radical feminist rant by Valerie Solanos published in 1967. Solanos, by the way, later shot the artist Andy Warhol, an act, driven by her developing paranoia. He was seriously wounded, requiring a surgical corset to hold his organs in place for the rest of his life. TNW takes a middle path, reporting the facts of the situation with little bias but using the system to have me voice very vulgar phrases. (...)

The technology for destruction and abuse is already here. The technology to counter that... is not.

https://www.youtube.com/watch?v=adGrQ-fq570

Edit: After Jordan Peterson started procedures for a Lawsuit against the owners of the site, its functionality was taken out.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on August 22, 2019, 07:34:30 PM
I'm tempted to say that Jordan Peterson said it, therefore it's worthless, but I won't. Instead I'll say this:

Yeah. Deal with it.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on August 23, 2019, 05:43:43 PM
It's quite impressive, the extent to which these wealthy, overpaid bigots have made a cottage industry of dreaming up novel ways to portray themselves as victims.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on August 29, 2019, 03:06:26 PM
From Nature:
Halt the use of facial-recognition technology until it is regulated
https://www.nature.com/articles/d41586-019-02514-7
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on September 04, 2019, 06:06:30 PM
And then... this happened. We can be sure... this is a first of many, MANY MORE to come. The general population is training AIs because of memes... and this is the result.

Deepfaked Voice of CEO Used to Steal Almost $250,000 from Company
https://interestingengineering.com/deepfaked-voice-of-ceo-used-to-steal-almost-250000-from-company
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on September 04, 2019, 08:47:35 PM
Theft and embezzlement is already a crime. The only thing that has changed here is the method.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: stochastibot on September 06, 2019, 03:32:27 AM
It seems that DARPA is already funding research on detecting faked videos:

https://www.technologyreview.com/s/611726/the-defense-department-has-produced-the-first-tools-for-catching-deepfakes/

and FAIR is actively looking into it as well:

https://ai.facebook.com/blog/deepfake-detection-challenge

Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: stochastibot on September 06, 2019, 03:41:15 AM
I found a few papers reporting around 97% - 99% accuracy in detecting faked videos, so I don't think faked videos are going to be that huge of a threat. 
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: arthwollipot on September 06, 2019, 04:02:48 AM
I found a few papers reporting around 97% - 99% accuracy in detecting faked videos, so I don't think faked videos are going to be that huge of a threat.

But now begins the arms race.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on October 12, 2019, 12:36:41 PM
Well.... the day arrived.

1) A study showed that people already are as good as a coin flip in terms of saying if certain videos are deepfakes or not. This is the milestone of the complete failure from humans to accurately tell fiction from reality.
2) This awesome detection technology will solve all the deepfake problems, right? Alas.... it will just accelerate it further. This algorithm will now be used as the Discriminator on the Generative Adversarial Network, and Deepfakes are now asymptotically getting close to perfection. We already have two feet on dystopic territory.

DeepFake Detector AIs Are Good Too!
https://www.youtube.com/watch?v=RoGHVI-w9bE

(https://i.imgur.com/oEZO53X.jpg)
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: The Latinist on October 12, 2019, 01:28:34 PM
I fail to see the point of wringing our hands over something that is inevitable.  We must adjust to the new reality, just as we've adjusted to the reality that privacy is an illusion.  The alternative is a Butlerian jihad, which I don't think any of us wants.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on October 12, 2019, 03:07:42 PM
https://youtu.be/5rPKeUXjEvE

I found this funny but mostly really impressive.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: Igor SMC on October 12, 2019, 04:10:21 PM
@brilligtove

Very impressive indeed!
Did you noticed that that video is "cheating" precisely in one of the ways mentioned in the paper above? They even placed an option of 4k resolution in the settings.... but the resolution of the video was purposefully scaled down to hide the artifacts from the image.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on October 12, 2019, 07:44:49 PM
At least we have Captain Disillusion on our side.

https://www.youtube.com/watch?v=HqjKn0iC6l8
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: brilligtove on October 12, 2019, 10:03:26 PM
@brilligtove

Very impressive indeed!
Did you noticed that that video is "cheating" precisely in one of the ways mentioned in the paper above? They even placed an option of 4k resolution in the settings.... but the resolution of the video was purposefully scaled down to hide the artifacts from the image.

I did not notice that - but I haven't had time to look at the paper. I am impressed by how effective the combination of voice and facial morphing can be, though.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: stochastibot on October 18, 2019, 04:56:15 PM
1) A study showed that people already are as good as a coin flip in terms of saying if certain videos are deepfakes or not. This is the milestone of the complete failure from humans to accurately tell fiction from reality.

Assuming you're talking about the FaceForensics++ paper, the human study only have people look at images, not videos, and for only 2 to 6 seconds per image.  If people had videos to look at, as opposed to relying on stitching and warping artifacts from a single image, I wouldn't be surprised if the numbers would end up being higher.

Quote
2) This awesome detection technology will solve all the deepfake problems, right? Alas.... it will just accelerate it further. This algorithm will now be used as the Discriminator on the Generative Adversarial Network, and Deepfakes are now asymptotically getting close to perfection. We already have two feet on dystopic territory.

The first detector used some 2012-era steganalysis to detect fakes, which wouldn't be able to be used as a discriminator in a GAN.  What remains unexplored is the use of detectors that exploit temporal aspects of the videos.  I'd bet that there's a reasonable number of approaches that could detect video manipulation that the GANs that deepfakes use don't.
Title: Re: All Deepfake research must be terminated. Society is not ready.
Post by: John Albert on October 21, 2019, 05:18:23 PM
Impressionist + Deep Fake tech

https://youtu.be/W8DcLaPLsMk