Teen boys use AI to make fake nudes of classmates, sparking police probe::Parents told the high school “believed” the deepfake nudes were deleted.

  • LogicalDrivel@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    123
    arrow-down
    1
    ·
    1 year ago

    “…and their principal, Asfendis, has vowed to raise awareness on campus of how to use new technologies responsibly.”

    Surely all the teenage boys will understand, and only use the technology for wholesome purposes.

  • Mac@mander.xyz
    link
    fedilink
    English
    arrow-up
    110
    arrow-down
    4
    ·
    1 year ago

    There is absolutely no way anyone could have possibly seen this coming.

      • surewhynotlem@lemmy.world
        link
        fedilink
        English
        arrow-up
        97
        arrow-down
        5
        ·
        1 year ago

        You don’t. Scissors and Polaroid and Playboy have been around for decades. If you wanted to see your classmates face on a nude and photocopy it, you could.

        Now it’s just easier and more believable. But it’s not any more stoppable.

        • RememberTheApollo@lemmy.world
          link
          fedilink
          English
          arrow-up
          9
          arrow-down
          1
          ·
          1 year ago

          Tbf the ease of creating multiple realistic images quickly along with the ability to rapidly share those images is a bit different than cut ‘n paste a completely unmatched head and torso.

          • kent_eh@lemmy.ca
            link
            fedilink
            English
            arrow-up
            9
            arrow-down
            1
            ·
            1 year ago

            I grew up “back then” (when VHS was new technology) and it really wasn’t very hard to get some sort of porn.

            • XIIIesq@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              1
              ·
              1 year ago

              Even as a minor? I grew up way later than that and the only way we’d get porn before the internet really kicked off was to get lucky and find an old mag in a bush.

              • Buddahriffic@lemmy.world
                link
                fedilink
                English
                arrow-up
                4
                ·
                1 year ago

                I grew up with internet porn, but it was also during the transition so I had magazines as a minor also. It was just as easy to have some, the only difference now is the volume that’s easy to access. If people had physical porn, it had to be physically hidden, and could then be physically found. And if you’re always buying more, it gets harder and harder to both hide it and keep track of what you have so that you’ll notice some missing.

                Though my first memories of porn are from going to a corner store style store in a mall and just looking through the porn magazines they had on display. A lot of the employees running the stores didn’t give a shit. Maybe they wouldn’t have sold them to me, but they either didn’t even notice or just didn’t say anything when I was looking at porn before I even knew about jerking off.

              • kent_eh@lemmy.ca
                link
                fedilink
                English
                arrow-up
                2
                ·
                1 year ago

                Even as a minor?

                Yup. Neighbour’s kid’s dad kept his Hustler stash in their garden shed in a crate behind the lawn mower.

          • XIIIesq@lemmy.world
            link
            fedilink
            English
            arrow-up
            4
            arrow-down
            1
            ·
            1 year ago

            Yh, I don’t like that my generation was the first to be exposed to freely available and copious amounts of porn. But on the flip side the internet is sooooooo useful and I would not endorse any government saying what should or shouldn’t be on the internet, the internet should be free and censorship would be a very slippery slope.

            It’s a tough moral debate and I’m really not sure what the answer is.

            • r3df0x ✡️✝☪️@7.62x54r.ru
              link
              fedilink
              English
              arrow-up
              3
              ·
              edit-2
              1 year ago

              In general, I’m against “censorship.” There’s also no reason why we should allow non consensual porn to be circulated. Abolishing online porn entirely would instantly solve the question of “revenge porn” and whether consent was obtained.

              There’s a certain amount of logic to the idea that you should allow bad people to feel free to express themselves so they can be identified, but I don’t think that fully holds up with porn and can lead to women being harmed. Girls who are 18-22 aren’t in a position to fully resist the temptation being offered by a lot of sex work possibilities. They haven’t learned about money yet and getting offered “a lot” in the short term is going to be too hard for them to weigh against selling out their future at that age. For every “empowered sex worker” out there who makes a good living and really wants to be in that work, there are many more who were exploited or got into it because of mental illness or trauma. Commodification of sex is ultimately a feature of the capitalist system.

            • idunnololz@lemmy.world
              link
              fedilink
              English
              arrow-up
              8
              arrow-down
              9
              ·
              1 year ago

              There might be a misunderstanding. I understand the original post is trying to say that it was obvious problems like this will occur with the introduction of AI generated images but it also implies an easy or obvious solution. But there isn’t one, so what is the point of pointing this out.

              • BlueÆther@no.lastname.nz
                link
                fedilink
                English
                arrow-up
                10
                arrow-down
                1
                ·
                1 year ago

                I don’t read it as saying that there may be a simple solution? And I don’t know how to attack the problem other that maybe a posable threat of distribution of material that could be classed as CSAM

                • idunnololz@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  4
                  arrow-down
                  3
                  ·
                  edit-2
                  1 year ago

                  Maybe an analogy would help clear this whole thread up. Let’s say you wake up tomorrow and you see headlines of scientists discovering a meteor that will hit earth in the next 48 hours. Then a couple of days later you read a meteor hit earth causing X deaths and Y billions of dollars in damages. Then you go to the comment section and read “There is absolutely no way anyone could have possibly seen this coming.” So then you’re thinking to yourself does this comment seem a bit weird or am I just dumb for missing something. So you ask “could this have been prevented somehow” (subtext you don’t really see anything obvious) but then you get confirmation it could not have been prevented so now you’re just like “wait then wtf was the original comment saying”.

                  And that is how I feel right now lmao.

      • saltesc@lemmy.world
        link
        fedilink
        English
        arrow-up
        12
        arrow-down
        4
        ·
        edit-2
        1 year ago

        Go back to using natural intelligence and try render with brain. Images can’t be shared.

        • Melt@lemm.ee
          link
          fedilink
          English
          arrow-up
          9
          arrow-down
          3
          ·
          1 year ago

          Brain sends data to hand and hand render it with pen and paper, what now?

        • Kusimulkku@lemm.ee
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          1
          ·
          1 year ago

          This reminds me of the funny picture about a black person being angry that white people can think of slurs and there’s nothing that can be done about it

  • Otter@lemmy.ca
    link
    fedilink
    English
    arrow-up
    76
    ·
    edit-2
    1 year ago

    The other comment about how this has been happening for a long time (with low tech methods) is true, and it’s also true that we can’t stop this completely. We can still respond to it:

    An immediate and easy focus would be on what they do with the images. Sharing them around is still harassment / bullying and it should be dealt with in the same way as it currently is.

    There’s also an education aspect to it. In the past, those images (magazines, photocopies, photoshop) would be limited in who sees them. The kids now are likely using free online tools that aren’t private or secure, and those images could stick around forever. So it could be good to highlight that

    • Your friends and classmates may see them, and it may harm their lives. The images will likely stick around. Facial recognition algorithms are also improving, so it’s a legitimate concern that an image stored on a random site somewhere will be tied back to them.
    • The images can be traced back to the creator and the creator can face repercussions for it (for those without empathy, this might be the better selling point
    • Emma_Gold_Man@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      23
      ·
      1 year ago

      Your point 1 seems to forget something important: kids are often cruel, and bullying is frequently the point. So long term consequences for their classmates can be an incentive more than a deterrent.

      • Buddahriffic@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        Yeah.

        What teacher says: You shouldn’t do this because it might hurt somebody.

        What some kids hear: Check out this new way to hurt somebody and get horny at the same time. And as an added benefit, you can say it’s only about the first one if admitting the second one would hurt you socially, even if the second one was the whole original point!

    • Adalast@lemmy.world
      link
      fedilink
      English
      arrow-up
      18
      arrow-down
      3
      ·
      1 year ago

      To your first point, much to the benefit of humanity, and counter to popular belief, the internet is NOT forever. Between link rot, data purges, corporate buyouts, transmission compression losses, and general human stupidity, large swaths of the internet have vanished. Hell, just Macromedia selling out to Adobe ended up causing the loss of most of the popular internet games and videos for anyone in their mid to late 30s at this point (you will be misses Flash). The odds of these specific AI-generated child porn pictures surviving even in some dark corner of the bright web are slim to none. And if they end up surviving in the dark web, well, anyone who sees them will likely have a LOT of explaining to do.

      Also, for the commentary of the websites keeping the images. That is doubtful, beyond holding them in an account-bound locker for the user to retrieve. They don’t care and too many images get generated every day for them to see it as more than reinforcement training.

      Speaking of reinforcement training, they may have been able to use Photoshop’s new generative fill to do this, but to actually generate fresh images of a specific peer they would have had to train a LoRA or Hypernerwork on photos of the girl so the SD could actually resolve it. They weren’t doing that on an AI site, especially not a free one. They were probably using ComfyUI or Automatic1111 (I use both myself). They are free, open source, locally executed software that allow you to use the aforementioned tools when generating. That means that the images were restricted to their local machine, then transferred to a cell phone and distributed to friends.

      https://www.theatlantic.com/technology/archive/2021/06/the-internet-is-a-collective-hallucination/619320/

    • cy_narrator@discuss.tchncs.de
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      9
      ·
      1 year ago

      I think we should pressure EU to make it such that any online AI photo generating website also uses AI to make sure what was asked is not illegal.

  • 6daemonbag@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    50
    ·
    1 year ago

    My niece had this same issue a few years ago but with Photoshop. It absolutely ruined her. Changed schools multiple times (public and private) but social media exists so all the kids knew. She ended up getting homeschooled for the last 5 years of school as well as a fuckload of therapy. She came out the other side okay but she has massive trust issues and anxiety

  • DasherPack@lemmy.world
    link
    fedilink
    English
    arrow-up
    28
    arrow-down
    6
    ·
    1 year ago

    In Spain it happened recently with some 12y/olds…it created a country-wide debate, and as always, did not lead to any regulation. Hopefully the EU will do something

  • AutoTL;DR@lemmings.worldB
    link
    fedilink
    English
    arrow-up
    11
    ·
    1 year ago

    This is the best summary I could come up with:


    This October, boys at Westfield High School in New Jersey started acting “weird,” the Wall Street Journal reported.

    It took four days before the school found out that the boys had been using AI image generators to create and share fake nude photos of female classmates.

    Biden asked the secretary of Commerce, the secretary of Homeland Security, and the heads of other appropriate agencies to provide recommendations regarding “testing and safeguards against” producing “child sexual abuse material” and “non-consensual intimate imagery of real individuals (including intimate digital depictions of the body or body parts of an identifiable individual), for generative AI.”

    “New York State currently lacks the adequate criminal statutes to protect victims of ‘deepfake’ pornography, both adults and children,” Donnelly said.

    Until laws are strengthened, Bramnick has asked the Union County prosecutor to find out what happened at Westfield High School, and state police are still investigating.

    Until the matter is settled in the New Jersey town, the girls plan to keep advocating for victims, and their principal, Asfendis, has vowed to raise awareness on campus of how to use new technologies responsibly.


    The original article contains 950 words, the summary contains 184 words. Saved 81%. I’m a bot and I’m open source!

  • randon31415@lemmy.world
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    1
    ·
    1 year ago

    Pictures? We are on the edge of believable videos with AI produced voices and sounds - made on normal computers. Need to clear a few more hurdles in 3D AI modeling, VR, and haptic feedback before this trend reaches it’s obvious conclusion.

    Wonder what crime it would be called if you create a haptic VR double of someone unconsensually and don’t distribute it?

    • jackalope@lemmy.ml
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      3
      ·
      1 year ago

      Haptics are never going to be like in Ready Player One. It’s crazy to me that anyone believes the tech will be capable of that. Like how diminished is one’s sense of touch that one could believe it could be fooled by fancy rumble packs? Touch is so much more complex than that. Piezoelectric motors vibrating are not going to be able to be able to fake solidity. Nuts to me people think that.

      • TheDarksteel94@sopuli.xyz
        link
        fedilink
        English
        arrow-up
        4
        ·
        1 year ago

        Might be possible with big gel tanks that people get submerged in, so the gel would be somehow hardened or softened with precise and weak electric currents, emulating textures.

        But imo, it’s more likely that it’ll happen through some brain interface and the whole experience will basically be a very lucid dream.

        Lots of time until that though, unless we destroy ourselves first. At least I doubt it’ll happen during my lifetime.

        • jackalope@lemmy.ml
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          1
          ·
          1 year ago

          It seems more likely to happen through a brain interface, but also I’m increasingly skeptical that will ever be possible. Optimistic estimates for a full brain interface are a century plus, just by judging at the number of direct neuron measures we currently have and applying a (optimistic) Moore’s law style exponential curve: https://waitbutwhy.com/2017/04/neuralink.html

          Vr is real fun but it’s fundamentally just another display technology. It’s less “ready player one” and more “what 3d TVs promised and failed to be”.

          • TheDarksteel94@sopuli.xyz
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 year ago

            True that. I just can’t wait until there are full headsets that are as small as glasses with wireless data transmission to my PC. There are at least a few companies that are coming closer every year, like Meta. Not a big fan of Meta though.

      • time_lord@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        1
        ·
        1 year ago

        Like how diminished is one’s sense of touch that one could believe it could be fooled by fancy rumble packs?

        Have you ever used a macbook trackpad? The click is just a fancy rumble pack. We can use electricity to make glass opaque. If the only thing stopping a person from living in a VR pod is haptic feedback, it’ll be solved in a fortnight.

        • jackalope@lemmy.ml
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          1
          ·
          1 year ago

          Then why hasn’t it been solved? It’s been nearly a decade since the oculus sdk came out.

          And if you think the max track pad haptics are indistinguishable from a real button click, you’re… Not very perceptive imo. Don’t mean that as an attack. Just open your mind to the idea that other people can def tell the difference.

  • mwguy@infosec.pub
    link
    fedilink
    English
    arrow-up
    6
    arrow-down
    2
    ·
    1 year ago

    Given that AI images and media can’t be copyrighted, does the nominal “subject” have any recourse?

    • kadu@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      ·
      1 year ago

      Not sure about other places, but here in Brazil creating a fake nude of somebody and distributing it would be illegal

      • mwguy@infosec.pub
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        2
        ·
        1 year ago

        In the US a photoshopped nude would be copyrightable. But courts here have said that AI generated content doesn’t get the benefits of copyright.

        • essteeyou@lemmy.world
          link
          fedilink
          English
          arrow-up
          4
          ·
          1 year ago

          Don’t only think of copyright. People don’t copyright CSAM, but they go to jail for making/distributing it.

        • CileTheSane@lemmy.ca
          link
          fedilink
          English
          arrow-up
          4
          ·
          1 year ago

          Not being able to apply for copyright doesn’t prevent you from getting charged for infringement.

          “I made it with AI, it’s not copyrightable” is the 2023 version of uploading a show to Youtube and adding in the details “I do not claim Copyright on this material. All Copyrights belong to their respective holders”. It’s still illegal even if you don’t claim to own it.

    • time_lord@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      There’s the matter of consent, and it might legally be along the same lines of giving someone a roofie so they don’t remember in the morning.

    • oatscoop@midwest.social
      link
      fedilink
      English
      arrow-up
      0
      ·
      edit-2
      1 year ago

      That just means the person that used AI to make something can’t claim those rights for the generated content – other laws still apply.

      Everyone else still retain rights to their likeness in most places, and I’d imagine that still stands in this case.

      • mwguy@infosec.pub
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Historically, I know that a big way that the dissemination of these sort of images was stopped was by using copyright law (because they’re using the likeness of the subject). I’m worried how that will work if there’s no copyright law to fall back in.

  • badbytes@lemmy.world
    link
    fedilink
    English
    arrow-up
    6
    arrow-down
    2
    ·
    1 year ago

    Thanks authorities, for taking seriously the safety of our students in the classroom. I also saw a kid eat glue.

  • r3df0x ✡️✝☪️@7.62x54r.ru
    link
    fedilink
    English
    arrow-up
    14
    arrow-down
    19
    ·
    1 year ago

    If you’re making porn of real underage people, I have no problem with you being put on the pedo registry.

    If no serious harm was done, I’m fine with convicting them and then doing full expungement after 5-10 years.

    • wildginger@lemmy.myserv.one
      link
      fedilink
      English
      arrow-up
      28
      arrow-down
      2
      ·
      1 year ago

      And youre proof that the pedo registry shouldnt exist as is.

      Teenagers being sexually interested in their peers is not pedophilia, and you want to ruin a decade of their life guaranteed, with the “”“”““promise””“”“”" of an expungement that would never actually happen thanks to the permanent nature of the internet for it.

      This misuse of AI is a crime and should be punished and deterred, obviously. But labeling children about to enter the world as pedophiles basically for the rest of their lives?

      Youre kind of a monster.

      • r3df0x ✡️✝☪️@7.62x54r.ru
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        5
        ·
        1 year ago

        What about the fact that the girls who are victims of something like this will have to contend with the pictures being online if someone posts them there? What if people who don’t know that the pictures depict minors re-post them to other sites, making them very difficult to remove? That can cause very serious employablity problems. It doesn’t matter how open minded people are, they don’t want porn coming up if someone googles one of their employees.

        • wildginger@lemmy.myserv.one
          link
          fedilink
          English
          arrow-up
          8
          arrow-down
          2
          ·
          1 year ago

          They are children. Being horny about classmates.

          Being sexually aroused by people your own age and wishing to fantasize about it is not enabling pedophilia, you literal psychopath.

          • r3df0x ✡️✝☪️@7.62x54r.ru
            link
            fedilink
            English
            arrow-up
            4
            arrow-down
            8
            ·
            1 year ago

            Circulating porn of minors is a crime and enables pedophiles. Not to mention teenage girls could easily commit suicide over something like this.

            • Fades@lemmy.world
              link
              fedilink
              English
              arrow-up
              5
              arrow-down
              2
              ·
              edit-2
              1 year ago

              So does yearbook and any other kind of photos that depict children for that matter

              You can’t keep pushing the goal posts, by your logic young people should never date or take photos together because it could enable pedophiles somewhere somehow

              These are children with brains still in development, they are discovering themselves and you want to label them forever a pedophile because they didn’t make a conscious effort to research how their spanking material could potentially enable a pedo (because we all know pedos can only be enabled by things produced by kids… yeah that’s the real threat)

              Instead of suggesting a way to help the victims you are advocating of the creation of yet more victims

              What a pathetic brain dead stance you are defending

              • eatthecake@lemmy.world
                link
                fedilink
                English
                arrow-up
                2
                ·
                1 year ago

                Abuse and bullying of their classmates is just ‘discovering themselves’? Discovering that they’re psychopathic little mysoginists I guess. Their ‘spanking material’ was created in order to demean and huumiliate their victims. There’s plenty of porn online and absolutely no need for them to do this. If you actuslly wanted to help the victims you would not be trivialising and excusing this behaviour as ‘being horny about classmates’.

                • wildginger@lemmy.myserv.one
                  link
                  fedilink
                  English
                  arrow-up
                  2
                  arrow-down
                  1
                  ·
                  1 year ago

                  And an AI image with a face photoshopped over it isnt a photo of a child.

                  And a teen being sexually interested in other teens isnt a pedophile.

    • Jolteon@lemmy.zip
      link
      fedilink
      English
      arrow-up
      13
      arrow-down
      1
      ·
      1 year ago

      I’d argue that someone making porn of someone their own age is not pedophilia.

    • XIIIesq@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 year ago

      That’s an easy enough judgement when the perpetrator is an adult. What do you do when the perpetrator is a minor themselves? As they are in this article.

      Of course their still needs to be some sort of recourse, but for every other crime there is a difference between being tried as a child or being tried as an adult.

      I find it tough to consider myself.

      • r3df0x ✡️✝☪️@7.62x54r.ru
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        4
        ·
        1 year ago

        Considering the consequences for a high school student if porn of them gets circulated, I’m fine with putting them on the registry. Expungement can happen later based on the aftermath. Teenage girls have killed themselves over this sort of thing.