One of the leaders of the private messaging app Signal calls the feature 'dangerous' because it could potentially spy on users.

Google’s AI model will potentially listen in on all your phone calls — or at least ones it suspects are coming from a fraudster.

To protect the user’s privacy, the company says Gemini Nano operates locally, without connecting to the internet. “This protection all happens on-device, so your conversation stays private to you. We’ll share more about this opt-in feature later this year,” the company says.

“This is incredibly dangerous,” says Meredith Whittaker, the president of a foundation for the end-to-end encrypted messaging app Signal.

Whittaker —a former Google employee— argues that the entire premise of the anti-scam call feature poses a potential threat. That’s because Google could potentially program the same technology to scan for other keywords, like asking for access to abortion services.

“It lays the path for centralized, device-level client-side scanning,” she said in a post on Twitter/X. “From detecting ‘scams’ it’s a short step to ‘detecting patterns commonly associated w/ seeking reproductive care’ or ‘commonly associated w/ providing LGBTQ resources’ or ‘commonly associated with tech worker whistleblowing.’”

  • fruitycoder@sh.itjust.worksEnglish
    0·
    6 months ago

    As a OpenSource app with no need for centralized server it would be great. I want that. As spyware configured out of my control absolutely not.

    • GenosseFlosse@lemmy.nzEnglish
      0·
      6 months ago

      I think you do need some central server, and then check if a lot of users report certain number blocks for spam in a short amount of time. No need for AI on this one. Isn’t that how most phone spam blockers work?

  • ᴅᴜᴋᴇᴛʜᴏʀɪᴏɴ@lemmy.worldEnglish
    0·
    6 months ago

    Let’s talk about wiretapping laws and states where two-party consent is required to record a call.

    Where I live, I must notify the other party that I am recording. If not, it’s illegal. Also, any audio recorded without consent is not admissible in court.

    • Venia Silente@lemm.eeEnglish
      0·
      6 months ago

      Complete tangent but what is two-party consent even for? I can imagine it gets in the way of getting a lot of evidence in cases of domestic abuse or organized crime.

      Caller: [threatens me for 45 seconds] Me: “Could you call me again and repeat all that, for the recording?” Caller: [hangs]

      • ᴅᴜᴋᴇᴛʜᴏʀɪᴏɴ@lemmy.worldEnglish
        0·
        6 months ago

        Means both parties have to agree to be recorded (usually at the onset of a call).

        “Be advised, this call is recorded for quality assurance purposes” at which point you could hang up. The 4th Amendment still applies in America, regardless of what local cops and prosecutors believe. You have a right to privacy in two-party conversations.

    • Phoenixz@lemmy.caEnglish
      0·
      6 months ago

      not admissible in court

      I get what you’re saying there but that sounds very much like the kind of detail that s possible future Trump administration wouldn’t much care about

        • Phoenixz@lemmy.caEnglish
          0·
          6 months ago

          That is probably the dumbest thing you said all week. At least, I hope it is because i don’t want to know how much dumber than that you can get.

            • Phoenixz@lemmy.caEnglish
              0·
              6 months ago

              … Wut?

              Obama was authoritarian? I’m intrigued; Exactly what mental gymnastics are you using to get that idea in your head?

              Few simple facts. Obama served twice. First time he won, and he had to campaign again to be elected a second yime.Second time he couldn’t run again for the third time, so he didn’t.

              Where exactly do you see authoritarian? Fox news quotes not allowed for answers here, I don’t need sad lies. Simple facts, what did Obama do to be authoritarian?

              I can’t wait…

              • ᴅᴜᴋᴇᴛʜᴏʀɪᴏɴ@lemmy.worldEnglish
                0·
                6 months ago

                Ask how many civilians Obama killed without congressional approval under the War Powers Act.

                Google the IRS targeting conservatives under Lois Lerner and sanctioned by the Administration.

    • tmyakal@lemm.eeEnglish
      0·
      6 months ago

      “Not admissible because it was illegally captured” didn’t give me the warm-and-fuzzies this comment sounds like it should’ve.

    • kent_eh@lemmy.caEnglish
      0·
      6 months ago

      Assuming it works the way Google says it does,

      There was a time when I trusted Google to be open and honest.

      But that time is long in the past.

  • TWeaK@lemm.eeEnglish
    0·
    6 months ago

    I feel like this would require cooperation from the manufacturer, as Google doesn’t actually provide the Phone app (except when they are the manufacturer).

    • noodlejetski@lemm.eeEnglish
      0·
      6 months ago

      except that it does. some OEMs like Samsung serve their own (or at least Samsung used to; not sure if it’s still true), but it’s definitely available on non-Google-branded phones.

      • TWeaK@lemm.eeEnglish
        0·
        6 months ago

        It’s available on all phones, but they all have their own version, forked from long ago. Even the standard AOSP Phone app has long split from Google (who have ceased open source development of the app).

      • TWeaK@lemm.eeEnglish
        0·
        6 months ago

        But downloading an app won’t install it as a system app.

  • elrik@lemmy.worldEnglish
    0·
    6 months ago

    It might be a good feature for the elderly as long as it’s local and optionally enabled (especially if it can be enabled only for unknown callers).

    Yes, I understand you would never really know if it’s not always enabled. But then again, you currently don’t know if anything similar isn’t already enabled.

    For other users, again potentially useful if it’s opt in. However, many people (myself included) simply don’t answer the phone anymore unless it’s a caller we already know. I use Google’s call screening feature for any other caller not in my contact list already, and I would estimate about 1 in 20 or 5% of such calls I receive aren’t spam (marketing or fraud). Of those non-spam calls, the majority are appointment reminders I don’t need.

    So would I turn this feature on? No, I don’t have a need. Could it be beneficial for the elderly? Yes, but probably not implemented in a way where it would actually be effective.

    • theparadox@lemmy.worldEnglish
      0·
      6 months ago

      Honestly, if it was transparent enough, I’d be fine with a service you could turn on that would listen to the first x seconds/minutes of a phone call to try and detect an AI generated voice. I have family that would 100% wire thousands of dollars if someone with their kid’s voice asked them to.

      • elrik@lemmy.worldEnglish
        0·
        6 months ago

        I mean, yeah, exactly. Keep in mind scammers are targeting vulnerable people. Granted I don’t see how such a feature will work on my grandmother’s flip phone.

  • fubarx@lemmy.mlEnglish
    0·
    6 months ago

    One of the things they glided around was whether a lot of this on-device stuff needs a special processor chip with AI+security to work?

    The Pixel phones (especially newer ones) made by Google have them, but the vast majority of Android phones don’t.

    So either these features only work on latest Google phones (which will piss off licensees and partners), or they’re using plain old CPU/GPUs to do this sort of detection, in which case it will be sniffable by malicious third-parties.

    And let’s not forget that if the phone can listen to your conversation to detect malicious intent, any country can legally compel Google to provide them with the data by claiming it is part of a law-enforcement investigation.

    Things are going to get spicy in Android-land.

  • ᴅᴜᴋᴇᴛʜᴏʀɪᴏɴ@lemmy.worldEnglish
    0·
    6 months ago

    “…locally on device without connecting to the internet”

    How would it then report such behavior to Google, without internet?

    If it notifies the end user, what good does that do? My phone is at my ear, I don’t stop a conversation when another app sends a notification while I’m on a call.

    This will 100% report things in the background to Google.

    • GenderNeutralBro@lemmy.sdf.orgEnglish
      0·
      6 months ago

      There are a few ways this could work, but it hardly seems worth the effort if it’s not phoning home.

      They could have an on-device database of red flags and use on-device voice recognition against that database. But then what? Pop up a “scam likely” screen while you’re already mid-call? Maybe include an option to report scams back to Google with a transcript? I guess that could be useful.

      Any more more than that would be a privacy nightmare. I don’t want Google’s AI deciding which of my conversations are private and which get sent back to Google. Any non-zero false positive rate would simply be unacceptable.

      Maybe this is the first look at a new cat and mouse game: AI to detect AI-generated voices? AI-generated voice scams are already out there in the wild and will only become more common as time goes on.

    • The Hobbyist@lemmy.zipEnglish
      0·
      6 months ago

      You’re putting a very large amount of trust on something which may simply require the flip of a switch to add the specified information to be sent back to Google along with all the regular heavy telemetry already feeding back…