These companies should be criminally liable for training their models on child porn.

  • John@lemmy.ml
    link
    fedilink
    English
    arrow-up
    11
    ·
    4 hours ago

    We do not allow our child’s image online anywhere. School, sports, social media, whatever … no thanks.

  • deeprlyeh@lemmus.org
    link
    fedilink
    English
    arrow-up
    8
    ·
    5 hours ago

    This is a genie out of the bottle problem. People can just run their own models and do this.

    The problem with subscription models doing this is that they are fighting investigations and laws to keep doing it, not stop it.

      • deeprlyeh@lemmus.org
        link
        fedilink
        English
        arrow-up
        5
        ·
        4 hours ago

        If you train the mode yourself, then the only restrictions are what you put in place.

        Training models is easy. Especially if you only need it to do one thing.

  • FlashMobOfOne@lemmy.world
    link
    fedilink
    English
    arrow-up
    11
    arrow-down
    1
    ·
    edit-2
    7 hours ago

    Yes, they certainly are, and most of the major platforms are being used for this and it’s widely known. Copilot’s prompt reviewers reported it to Microsoft, per leaked documents. Grok has been known to be in use for this basically since its inception. Meta’s LLM is used for it, and they know.

    I can’t speak for Gemini, Claude, or ChatGPT, but it wouldn’t surprise me at all if they’re being used for it too.

    These platforms are unregulated by anything other than civil lawsuits (though some international governments are starting to move on it), so yes, they’re going to appeal to every demographic, including the worst of the worst, in a desperate bid to one day be profitable.

    There was one guy arrested for it recently, a state-level deputy AG, but that dude’s the exception. Hopefully there will be more to come.

    I researched this topic recently and made a video about it.

    • shalafi@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      ·
      14 minutes ago

      I’ve only used ChatGPT, but sexually speaking, it’s pretty locked down. Maybe you could manipulate it given some effort but I’m only using one or two prompts and I’m out.

  • Glitchvid@lemmy.world
    link
    fedilink
    English
    arrow-up
    51
    arrow-down
    3
    ·
    12 hours ago

    One concerning thing is that when these AI companies are pushed to crack down on this, they’ll retrain and bias their models to only recognize traditional “pornstar” body types as ‘adult’.

    This is a problem when they then get used as a mass content flagging and moderation tool — it’s not hard to imagine some adult woman taking nudes (as people do) and just because they don’t have a huge bust, getting auto flagged as having illegal material. Not to mention body types that aren’t even real, like the art made in furry communities.

    It’s not like there isn’t precedent, recently Discord banned a ton of people because its scanning system started throwing false positives, and that was only corrected because it made such a stink.

    • square@lemmy.zip
      link
      fedilink
      English
      arrow-up
      4
      ·
      3 hours ago

      There was a story a while ago where some dude was about to be convicted of possession for a few videos of a young looking woman. The prosecution even had an expert witness that repeatedly testified that it was impossible for an adult woman to look this way and wouldn’t be budged at all on cross examination. Luckily, at the 11th hour, the defense found the porn star, who was actually somewhat famous but the videos didn’t contain her stage name. She took the stand, a 20-something looking like a 13yo, and the case was dismissed.

      If this shit is automated there will definitely be a lot of false positives. And it will be automated, and it will be a “wide net” pulling in all kinds of stuff like you mentioned and more. And no one will really care to make it work better because “the children”. Meanwhile, the will be no additional effort to stop harm to actual children because…well, I think we all know why. Sloppy, performative, bullshit is what we’ll get instead of measures to prevent and help victims.

      • ayyy@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 hour ago

        This sounds like an urban legend, do you remember the time/location or any names for this case?

        • square@lemmy.zip
          link
          fedilink
          English
          arrow-up
          3
          ·
          edit-2
          37 minutes ago

          Yup, took a little digging though, I couldn’t remember the name off the top of my head. Back when this happened she would be a top result for “young Latina” on all the big sites, but I think the algorithms favour newer videos so I had to scroll several pages before I recognised her. If you want to read more about it Google “case dismissed Little Lupe.”

          I remember reading about this myself in a major news outlet specifically because I recognised the name at the time. Although she isn’t my type, absolutely no breasts and hips isn’t for me, if you use search terms for youthful actors, ones like this will also come up. Like I said, she was somewhat big at the time and I saw her name often.

          Here is another link with some more info. I did forget some details. The biggest one is the video was labelled with her name but the prosecutor refused to do a simple record search (all of her movies were compliant with the applicable laws regarding this), or accept all the evidence the defendant’s lawyer presented them with, and the defense had to fly her in to testify.

    • Virtvirt588@lemmy.world
      link
      fedilink
      English
      arrow-up
      9
      ·
      10 hours ago

      This is a problem when they then get used as a mass content flagging and moderation tool — it’s not hard to imagine some adult woman taking nudes (as people do).

      The problem is with sexual abuse material not nudity on its own, there is a clear distinction between those two. It seems like this is the major problem when it comes to “crack-downs” on this.

    • outlawcarl@fedinsfw.app
      link
      fedilink
      English
      arrow-up
      12
      arrow-down
      1
      ·
      12 hours ago

      Yeah we will drown in flase postives, my other worry is they will spend on this auto scaning of media call it problem solved slash bugets for other things and predators move to dumb tech or really encoded shit.

      Thats if there even is enouth money/compute to scan everything…

    • abadidonque@feddit.uk
      link
      fedilink
      English
      arrow-up
      5
      ·
      11 hours ago

      They should be made to retrain their models using only images of those palaeolithic venus figurines.

    • howdy@lemmy.ml
      link
      fedilink
      English
      arrow-up
      5
      arrow-down
      2
      ·
      11 hours ago

      AI shouldn’t train on porn at all. Too many ethical issues.

    • Snowclone@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      4 hours ago

      if you look up more information in the topic there’s been a huge surge in csam for several years not just right now. the info is out there.

  • Snowclone@lemmy.world
    link
    fedilink
    English
    arrow-up
    36
    arrow-down
    19
    ·
    13 hours ago

    https://tsjournal.org/index.php/jots/article/view/29

    exposure to csam, through self reporting, increases the risk the viewer will seek contact with children, in case you need more arguments against csam. I’ve heard the argument that AI generated images may be a harm reduction situation. it’s not. it’s incredibly damaging to the viewer if they are seeking it out, it makes them more likely to abuse, if they are exposed to it as victims it can cause lasting damage, much of it is just remixed images of actual child abuse victims, and that revictimizes them, and causes serious damage as well. We really need to stop pretending the companies that make this software aren’t responsible for this software.

      • Le_Mon2@lemmy.zip
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        9
        ·
        4 hours ago

        Well it’s different because video games aren’t created by comsuming and averaging out every piece of footage of children being raped that grok can find on the internet. Hope this helps.

      • MaggiWuerze@feddit.org
        link
        fedilink
        English
        arrow-up
        5
        ·
        10 hours ago

        I would argue, that in both cases it only (or at least only meaningfully) affects people who already have a leaning towards the specific issue.

      • Snowclone@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        4
        ·
        7 hours ago

        why would watching children be abused from real or stimulated sources be the same as playing a violent video game?

        • Lumisal@lemmy.world
          link
          fedilink
          English
          arrow-up
          14
          arrow-down
          1
          ·
          edit-2
          4 hours ago

          You’re right. A video game is far more interactive. Heck these days you can even put on a headset and really simulate the action of murdering people. Grand Theft Auto is a whole detailed simulation on muder, theft, and more. Wayyy more immersive than just plain video.

          Clearly then, by your own logic and the study’s conclusion, watching simulated sources make people more likely to do the thing after all. I mean, what sick person fantasizes about murdering strangers?

          Obviously we must ban the existence of violent video games or crime in society will spiral out of control.

          Edit: for the younger lemmings, this is sarcasm. “Fake X causes more real X” is really old propaganda, and its usually used on something most of the population already finds morally objectable + is harmful to society to advance some ulterior motive.

          In this case, probably government spying and the erosion of privacy, as usual.

          • Snowclone@lemmy.world
            link
            fedilink
            English
            arrow-up
            4
            arrow-down
            2
            ·
            edit-2
            5 hours ago

            your doing A LOT here, friend. I’m not sure I can unpack this… I’m not suggesting playing Grand Theft Auto is going to make anyone more likely to commit violent crimes. I don’t see the connection with csam. I don’t think the effects of viewing csam is equivalent or similar to playing a video game. I’ve played GTA, it’s really not an instruction manual or stimulation of detailed murder or violence, honestly, it’s a bit silly and while modern versions of the game are more detailed than the early versions of the game that were graphically more similar to Frogger than anything else, it’s still not very detailed. if you needed a murder equivalent to csam I think gore and ‘faces of death’ type of videos would be more comparable, video games are marketed to be fun and are rated and regulated pretty heavily in most areas of the world, AI generated csam isn’t being caught up in that same process of regulation or review, multiple corporations have made software that is producing AI generated csam and no one is really holding them accountable like you or me would be if we personally increased the reported rates of csam by huge orders of magnitude. we’d be in prison. the software isn’t free from the corporations that made it. they made it, they should be held accountable.

            • Lumisal@lemmy.world
              link
              fedilink
              English
              arrow-up
              10
              ·
              4 hours ago

              And you’re too young to remember back when the same talking points were used against video games to try to ban them or against movies and books “promoting” drugs. This isn’t a new thing. “Fake X causes more of Real X to happen!” Has being going on for decades if not over a century and has always been used for an ulterior motives.

              Making up statistics to find yet another way for governments to justify eliminating privacy isn’t going to save children or stop pedophilia, much like banning video games wouldn’t reduce gun deaths and banning drugs doesn’t really reduce drug use. The root cause is elsewhere.

              But you go ahead thinking your virtue signaling makes things better. I’m sure your name calling tactic will totally make a difference.

    • it_depends_man@lemmy.world
      link
      fedilink
      English
      arrow-up
      37
      arrow-down
      16
      ·
      12 hours ago

      This study explores a sample of 1,546 anonymous individuals

      I have a study of 23023012301230123050503450 anonymous individuals, of which 78% agreed that you smell like doo doo. Yep. True story. If you disagree, you hate science. How could you do this?

      I shouldn’t be this sarcastic, the journal is called " Journal of Online Trust and Safety (JOTS) " with a name like that, how could I ever be so arrogant to not believe 1546 anonymous individuals on the internet? Shame on me, really.

      • Virtvirt588@lemmy.world
        link
        fedilink
        English
        arrow-up
        12
        arrow-down
        1
        ·
        10 hours ago

        Agreed, studies on this topic are always what someone on the other end wants to hear. The premise is inconsistent and the conclusion doesnt hold. If this study was done about something less “taboo” it would’ve been lost and forgotten.

      • howdy@lemmy.ml
        link
        fedilink
        English
        arrow-up
        7
        arrow-down
        4
        ·
        11 hours ago

        reddit moment…

        you can’t just dismiss an argument because the study is small. Sure it’s small, but it’s not easy to get large data on this subject.

        The concept of csam exposure influencing behaviour is highly plausible.

        • howdy@lemmy.ml
          link
          fedilink
          English
          arrow-up
          1
          ·
          27 minutes ago

          Nice to see my comment get destroyed. I agree with all of you on the importance of being rigorous in science, and that self reporting surveys are not reliable.

          The reasoning that made me comment was: I think original commenter is making an argument in good faith. You can dismiss the study, but then at least argue the point? Surely everyone agrees that the opposite view – that AI generated csam could reduce harm – is insane???

        • Senal@programming.dev
          link
          fedilink
          English
          arrow-up
          6
          arrow-down
          1
          ·
          edit-2
          6 hours ago

          Aside from the other good replies you have around the subject, i’ll address something more fundamental to your argument.

          you can’t just dismiss an argument because the study is small.

          You absolutely can dismiss a study for being too small. In some cases you really should.

          If you can’t procure and retain a meaningful sample size, then you shouldn’t be running the experiment (or more usefully, designing an experiment that works with smaller sample sizes, or one that uses a demographic where you can get adequate sample sizes).

          It’s one of the reasons why participant gathering is usually done up front.

          • frongt@lemmy.zip
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            3
            ·
            7 hours ago

            True. But over 1500 responses is a massive sample size for a survey like this.

            I haven’t read the actual study yet, but I would assume they filtered out things like one person submitting a thousand responses.

            • Senal@programming.dev
              link
              fedilink
              English
              arrow-up
              3
              arrow-down
              1
              ·
              6 hours ago

              True. But over 1500 responses is a massive sample size for a survey like this.

              Genuine question, based on what?

              Like, I’m just seeing a number, I’ve no idea if it’s big or small in the context of this area of study.

              I’d be interested to know why that’s considered a good result.

              I haven’t read the actual study yet, but I would assume they filtered out things like one person submitting a thousand responses.

              Agreed, I’d assume a basic level of scientific rigour.

        • WhatAmLemmy@lemmy.world
          link
          fedilink
          English
          arrow-up
          16
          arrow-down
          1
          ·
          10 hours ago

          I agree with the plausibility of the concept, but anonymous surveys aren’t really something we should be calling a “study”. They’re barely considered evidence as they’re open to extreme errors, bias, and manipulation; especially by respondents.

        • it_depends_man@lemmy.world
          link
          fedilink
          English
          arrow-up
          21
          arrow-down
          5
          ·
          11 hours ago

          The concept of csam exposure influencing behaviour is highly plausible.

          The entire scientific community and publishing industry is a minefield. So is CSAM and the call for more restrictive governing of the internet.

          The thing I know for sure is that there are people on this planet who have a political and a financial motive to lie and fabricate studies to justify their politics.

          The thing I’m mostly skeptical on is the scientific process. Not because it can’t work, but because that entire field too, has bad incentives. “Reputation” is held up, frankly, by boomers who have trouble operating a printer, but did some important theoretical work before the internet was made public and by politics and a disinterested public that doesn’t apply scrutiny to what their “intellectual elite” is actually doing.

          The internet is a KNOWN source of bad information. Those 1546 “anonymous individuals” could be one bored 4chan user, or one chinese/russian/us spy agency worker justifying the next expansion of their budget.

          is highly plausible.

          Is not science. It’s guesswork.

          I appreciate that it is difficult to collect good information on this subject, but that’s just how it is. Tough luck. As a scientist, a proper scientist, you don’t get to publish “it’s plausible” from a source you can’t verify as fact.

          Second guessing science isn’t anti-science.

          • Snowclone@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            4
            ·
            edit-2
            6 hours ago

            wow it sounds like you hate surveys and science. sorry about that.

            my comment on an internet news article was that child sexual abuse material even if generated by AI isn’t a harm reduction, as it is shown to cause harm in multiple ways. if you want to argue that being exposed to csam as a viewer isn’t harmful to the viewer or doesn’t make them feel more inclined to abuse children in real life, that’s fine, you can do that AND hate science. it’s OK, man.

            I’m not trying to get you to agree surveys and science as a whole are valid.

            • Senal@programming.dev
              link
              fedilink
              English
              arrow-up
              4
              ·
              6 hours ago

              Questioning the validity of methodologies and results is a core part of the scientific method.

              “It has surveys, so it must be legit” is not.

                • Senal@programming.dev
                  link
                  fedilink
                  English
                  arrow-up
                  4
                  ·
                  5 hours ago

                  where did I make that claim?

                  wow it sounds like you hate surveys and science. sorry about that.

                  implying the points raised don’t have merit and are just the ramblings of a science (and survey) hater.

                  I’m not trying to get you to agree surveys and science as a whole are valid.

                  again, implying that somehow the points raised are only due to disagreeing with science (and again surveys for some reason)

                  especially when the rest of your reply is just restating your original claim and not responding to any of the raised points.


                  “this person must hate science and surveys” is the contextual equivalent of someone saying “hey perhaps we should not allow children to work 12 hour days in factories” and you replying with “this person must hate freedom”.

                  It’s completely ignoring the point and claiming victory anyway.

        • ohulancutash@feddit.uk
          link
          fedilink
          English
          arrow-up
          8
          arrow-down
          3
          ·
          edit-2
          9 hours ago

          Maybe not dismiss, but you can think twice before giving it much credit.

          The government is currently looking at making anti-semitism a zero-tolerance offence in schools, in the wake of the Bell Review, which showed that 41% of Jewish pupils had suffered anti-semitic abuse at school. These are indeed troubling numbers, but baffling as 70% of British Jewish pupils are in Jewish schools. Are nearly 20% of anti-semitic incidents really occurring in Jewish schools? Or is it a flaw in the study, which had a sample size of 92 children of which 70 responded?

          If we just need to be plausible, then why bother with the time and expense of studies at all? Sure, its plausible that antisemitism is on the rise in schools, but so are exclusions to a truly troubling all-time high. Is it really wise to add greatly to that number based on the experience of 29 study respondants? That isn’t even a classroom’s worth.

          Small data sets can severely distort results and need to be taken with that firmly in mind.