• 0 Posts
  • 37 Comments
Joined 3 years ago
cake
Cake day: May 14th, 2024

help-circle



  • I think the big takeaway here is that if you’re using Tor, or a VPN, or really the internet in general, you always need to think about MITM attacks. There are a lot of standard protections against exactly this kind of attack.

    …users who ignored warnings that the SSH host key had changed…

    SSH keys are one of those protections. Don’t ignore the warnings. That’s a big red flashing DANGER sign.

    …block ports … used to accept various forms of encrypted email—while continuing to allow ports for receiving unencrypted email. If email services downgraded the connections…

    TLS connections are another of those protections; don’t use apps or services that allow plaintext communication.

    I’d be interested to know which email clients will downgrade connections by default. That is an absolutely insane thing to do. I don’t think my email provider even accepts unencrypted connections anymore.

    This is scary but really, nobody’s supposed to trust Tor exit nodes in the first place. This doesn’t drastically change my comfort level with Tor. But it’s a good reminder that you need to be mindful of how your apps are configured.








  • It’s more than just that. Smaller models are improving rapidly and are already “good enough” for the majority of consumer use cases. It is entirely realistic that in a few years, there will be virtually no demand for trillion-parameter LLMs. Demand for inference might not shrink in terms of functionality, but it will absolutely shrink in terms of memory and compute requirements. And then there will be a hell of a lot more server capacity available than anyone needs or wants, because they’re over-investing like crazy now.

    What happens when hundreds of billions of dollars spent on datacenters basically goes *poof*?



  • Yeah, the business model is not sustainable. They’re playing with funny money for now. There are only a few ways for them to make real money: raise prices a ton, scale back model sizes, and focus on government/military contracts (i.e. steal from taxpayers).

    Oh, and enshittify. Enshittify like no one’s ever enshittified before.

    They’re already starting, on all fronts.

    OpenAI is raising prices like mad this year, and imposing stricter usage limits. They just basically cut the value of their $200/month plan in half while introducing a new $500/month plan. API costs also doubled for some models back in May.

    Recently there was an article saying how AI costs are falling by 13x per year: https://archive.is/U0Fju . So as time goes on, they’ll likely be able to offer “similar” levels of performance for drastically lower costs, and just pocket the difference.

    Today’s smaller models compare favorably with the biggest models from 2-3 years ago. ChatGPT 3.5 was a “big deal” in 2024, and now you can run a model that performs better on a decent laptop or even a high-end smartphone, basically for free. Small open models keep improving at a steady rate.

    They’re indirectly jacking up prices of electronics so much that consumer-level hardware is basically frozen in time, and in some cases even regressing. Look at Microsoft re-introducing 8GB Surface laptops after spending so much marketing money telling people they had to go up to a minimum of 16GB. If I re-built my PC today with the same specs, it would cost about double what I paid a few years ago. So maybe people actually won’t be able to run anything on their own hardware in the near future. Like Uber and AirBNB and Amazon before them, they could start to look comparatively better than the alternatives by destroying the entire fucking market around them. I don’t know how long that will hold though. It can’t stay like this forever…right? Right?!?








  • I don’t see a price there, so I’m not sure how “overpriced” it is. But it’s a class of product that is expensive across the board, compared to a typical desktop workstation or gaming PC.

    The use cases are mostly local AI. That’s very demanding and requires compromises one way or another. You want high memory capacity and high memory bandwidth most of all. You can’t get both without paying out the nose, so you need to find the sweet spot for your specific use cases.

    Discrete GPUs give you high bandwidth, but low capacity.

    Standard desktop memory gives you high capacity, but low bandwidth (and also low compute speed since CPUs are no good for the workloads).

    This type of product is a middle ground. Integrated memory gives you better bandwidth than DDR5, but not as good as a discrete GPU. Integrated GPU/NPU gives you much better performance than a desktop CPU, and can access the full integrated memory directly. Upgradeable memory is not really possible for this class of product.

    The real competition for this is a tricked-out MacBook Pro. Those are faster, but they ain’t cheap, and I don’t think Apple even sells any 192GB Macs anymore, and when they did it was close to $10k.