(this post figured on HNews here) This happened to me in 2020, and it has been on my mind again lately. During the worst of the pandemic, I found a vulnerability in a system that gave me access to the Brazilian federal…

145 points•perone•3 days ago•77 comments•

77 comments

Modified30192 days ago
Relevant:

https://www.reddit.com/r/YouShouldKnow/comments/1wssf4u/ysk_...

https://imgur.com/a/6FaQQhb (original post before being taken down by mods)

>The Work Number is an Equifax (who famously had a massive data breach a few years back) owned database with employment and pay information.

>You can create a login on theworknumber .com and pull your report. Mine is seventy four pages and had info from every company I've worked at in the last 13 years including every paycheck I had received with the exact dollar amount (both net and gross) and hours worked. It has employment start/end dates, termination reason, details about withholdings, benefit enrollment, union affiliation, etc etc etc.

>This data is sourced directly from the HR platform your employees use (ADP, Rippling, Gusto, iSolved, etc). Equifax sells your data for things like employment background checks.

>You cannot have your data removed from The Work Number. You can freeze your report (much like a credit report) but if a potential employer cannot access your frozen report that may disqualify you.

>Why YSK: If you're interviewing for a job you can be at a significant disadvantage especially for things like salary negotiations because they can literally see how much you make including your most recent paycheck. Creditors also can access these reports.

>This is the biggest privacy violation I have ever seen and almost nobody is even aware of it. Certainly none of us consented to having our employment and income data harvested and sold, especially since we get nothing in return. At a minimum, people should know about how this data can impact you.

>Edit: The Work Number is primarily for the US, but there are similar services for other countries.

tomrod1 day ago
I checked my data. It is either 100% wrong or misleading. Start and end dates, compensation amounts, benefits. Per the file, I've worked 4 years out of 20 in my career. As an employer I have never used them and I worry for any employer who does.
ciupicri2 days ago
From reddit:

> In my country (Hungary) you literally get a little pink book where your boss is supposed to sign the time you spent there. You keep it until retirement, the company holds it while you work there. It's also digital now but we're one foot in 1990 and one foot in 2016.

We had something similar in Romania too. Now it's 100% digital and no more delays in informing the government. The employeer needs to notify the government before someone starts work. With the paper version you had a couple of days even weeks if I'm not mistaking.

monster_truck2 days ago
I was hoping this would give me an exhaustive log of my work history but it has roughly 3% of my career, I froze it anyways
burner4200422 days ago
It's been awhile since I last checked mine. I noticed my corporate work history was all there, at the level of detail OP mentioned. Where as the warehouse, construction, and bartending jobs were not.

The other one to check is LexisNexus, which covers auto insurance coverage and traffic violations. In the US, gov't tracked traffic citations fall off after 5 years, but last forever in the baby credit bureaus we're discussing.

Also, INTERPOL. You never know...

Modified30192 days ago
Yeah I imagine how much it has will vary wildly from person to person, but some it has pretty much everything down to anal circumference.
throwawayfifo1 day ago
reddit is awful. The post has been removed along with multiple highly upvoted comments. I cannot see the original post.
RobotToaster2 days ago
It would be interesting to see how they would handle a GDPR request from a EU citizen who was employed by a US company.
mikewarot1 day ago
Every since the OPM hack of 2015, I've been apparent to me that my former field of IT administration has lost the plot. Nobody knows what a data diode is, or why you would use one. Systems that should clearly be air-gapped aren't.

While it's easy to lay this at the feet of AI getting better at hacking. I see it as an primarily an IT issue. We've collectively ignored the lessons of history, and made do with patch jobs over poorly chosen operating systems instead.

--

We need air gaps, data diodes, and capability based operating systems. Now that I'm retired, when I have the energy, I'm working on the data diode part.

This weeks lesson for me, personally, as I try to build an open source data diode, is that the Waveshare RP2350-ETH is a horrible choice for a proxy/data source/sink, as the CH9120 ethernet interface can't do promiscuous mode. It might still be sufficient to build a data diode that can mirror a website, with << $50 component cost. Time will tell.

jandrewrogers1 day ago
In the last few years I’ve seen multiple startups describe normal systems running on AWS as “air-gapped” because there is a firewall. LARPers have diluted several terms that had specific meaning in a system isolation and security context to the point that I can no longer trust when people describe their systems using them.

There are far too many unserious people representing our industry.

bunderbunder1 day ago
I suspect that part of the problem here is that PaaS cloud deployments are fundamentally incompatible with some of these security measures. Putting everything into a big slushpool of compute that you dynamically reconfigure at the software layer via an Internet service pretty much precludes the use of true data diodes and air gapping.

But companies still want to sell products, including to people who are at least nominally concerned about security, and marketing's gonna market.

andwur1 day ago
Add in the suite of novel attachments of the prefix "cyber" to further muddy the water. The cyberfence will deter the cybertheives from accessing the cybercloud cybersuite we have just cybersold you (down the river on).

Competence isn't widely valued, obedience and sales figures are. Which wouldn't be so problematic if evolution could run its course and eliminate incompetence naturally, but that's now hard to see coming to pass when we have towering circular supply chains that feed on it.

m4631 day ago
> Systems that should clearly be air-gapped aren't

Personally I think refrigerators, microwaves, washing machines, vacuum cleaners, ...

I think at some point people just give up or the (bad) idea gets normalized.

Then at work, they're not surprised when some other system is connected for convenience...

EvanAnderson1 day ago
> Systems that should clearly be air-gapped aren't.

I'd take systems that were permitted to have filtered egress at this point. The lion's share of my work is in networking, so network segmentation is the "hammer" I pick up first.

Vendors gnash teeth and complain when I ask for their app's dependencies on hosted APIs and off-site resources. In the environments where I'm mandated to maintain FBI CJIS compliance I can still hold vendors accountable and get what I want. It's pretty much a lost battle in every other environment and unfiltered egress to the Internet from servers is just expected.

That's not even to get into the topic of communication flow within an application. >sigh<

wat100001 day ago
I would argue that the industry is learning the lessons of history quite well. The problem is that the lessons of history are that data breaches aren't really a problem for the breached organization. You get some bad press, maybe pay a pittance for identity theft monitoring, and then move on with your business. Why put effort into preventing something so inconsequential?
daveguy1 day ago
I'm curious what function a data diode (unidirectional network) would provide besides sensor data/mirroring/replication. Once you do anything that requires packet confirmation (TCP) you open yourself to OSI layer 4 security risks. Seems like mirroring/replication would need some feedback and then is it just sneakernet restore? Or another diode out from the replication for specific processing? If the same control system has access to both diodes then it's not a unidirectional system anymore. Is a data diode more of a pseudo-unidirection where it is enforced above TCP?

I completely agree that IT admin could be a lot more secure by design. Combined with better interfaces for responsible configuration.

theamk1 day ago
Not OP, but yeah, the data diodes need special software, you can't just proxy regular internet protocols over them. The way I've seen it, some use cases are:

- For ingress, you use special "file transfer" software. Run the receiver on secure side, run the sender on insecure side. It blasts the file "blind" - it has has no way to know if anyone ever received it. Make sure the receiver is fast enough, and the error-correcting codes are a great idea too, as they don't need feedback. It's up to user to want to secure side computer and check that the file was received without problems. Yeah, this is similar to sneakernet, but more secure, as you can't accidentally carry a virus on seemingly-empty drive.

- For status egress, you have secure side broadcast status periodically, say every minute. Insecure side receives the status, updates the database, and runs the regular web server to share the status. Again, secure side has no way to know if someone is listening on the other end, it just blasts out the messages and it's done.

And you are correct, if the secure system has both egress and ingress diodes, it is no longer isolated, and devious enough malware can establish two-way communications. But even if it won't save you from Stuxnet, the simple fact that it is no longer possible to have direct network connection to the outside raises security bar quite a bit - all the ideas about "let's just open this one port on firewall, it'll fine I swear" are completely stopped.

(Which reminds me of something in GP's (mikewaro) message: _why_ would a data diode need a promiscuous mode? Given every single data diode I have seen needs a special software on both sides, you should not need anything beoynd a basic TCP session)

dezgeg1 day ago
Not to mention how any sort of TLS key exchange is supposed to happen with an unidirectional network.
pjc501 day ago
"What will happen to the many government systems that will never get the chance to be AI-pentested?"

Oh, everybody's going to get AI-pentested whether they want to or know about it or not. It's the cost of being on the Internet. Probably the situation will continue to deteriorate. Both the British Library and Jaguar Land Rover recently suffered long outages due to compromises, for example. I suspect we'll probably just lose a few large, famous businesses entirely to compromises.

philipwhiuk1 day ago
Businesses will buy insurance priced slightly lower than the cost of hiring sufficient security as to not need the insurance.

(Many already do).

EvanAnderson1 day ago
Until "cyber" insurance underwriters suffer catastrophic losses and the price for policies rises accordingly I don't think we'll see any significant change in IT security status quo. If doing the right thing is more expensive than insuring away the risk the business will always make the cheaper decision.
teiferer2 days ago
> I honestly don’t blame them: [...] software is software

That attitude is the problem. Why does our industry have that attitude towards quality? Every bike shop in my little town is better with quality than the average software shop in the world. Yes, software is more complex than bicycle. But a software engineer also gets paid 10x and has the luxury of spending substantial time on their product, compared to the 10 minutes it takes the bike guy down the street to diagnose and then fix an issue with my bike which I then trust my life with once they are done and I bike through traffic.

We need to treat software differently. "Oh well, it's just software shrug" does not cut it anymore, if it even ever did.

vjk8001 day ago
Quality does pay off, however.

You could say that the entire selling point of Apple is that it's better quality than the competitors (even though it's still pretty bad if you compare it to a bicycle).

I guess it's more difficult to spot quality when it comes to software. I would definitely pay for high quality software. It's just that usually the crap quality of a software I'm using only shows slowly over time in little bugs and bad usability and there's no way for me to check for it beforehand.

graemep2 days ago
The customers care about the quality of a bike repair. They will pay more for a better repair, and will not return to someone who does a shoddy repair.

The owner, and probably user, of a bike is the person paying for the repair. It is to their advantage to ensure it is a good repair.

People expect a repair will be good, and will blame the person who repaired it if not. With software people often blame themselves for issues, and they have no expectation of quality. They also cannot tell quality until after they are committed to using it. Bike shops do not benefit from vendor lock-in.

frereubu1 day ago
> Yes, software is more complex than bicycle.

In ethical terms I agree that software quality should be better, but I think this line is too dismissive. Software is a few orders of magnitude more complicated than a bicycle, to the point where I would say it's an entirely fruitless comparison.

maeln2 days ago
> Why does our industry have that attitude towards quality? Every bike shop in my little town is better with quality than the average software shop in the world.

Personally, I find that we have been flooded with poorly designed, poorly QA'd, break-after-you-use-it-thrice product. I really don't think we have better quality standard in other industry. Quality, testing, design, etc has a cost and most company rather pull out a new version of their product every quarter than actually make a good product. The exception, which goes for software as well, is life critical applications (most public transit, defense, etc).

TheCapn1 day ago
I have, off and on, for years written diatribes about how if people in our industry want to run around with the "Engineer" moniker at some point they should start acting like it. That means respecting all aspects of what it means to be an Engineer and holding public well-being paramount is part of that. Licensure is part of that. Accountability and Responsibility is part of that.

I'm usually downvoted. The move fast and break things group seems to be blind (or willfully ignorant) to the harm that software systems can carry on a person's life. I don't know if its fear of regulation, or self-serving interests or some sort of disdain for the idea that we need to slow down and be more accountable to the public or what. At times the people I've engaged with lament on how they're capable but would be barred for lack of formal education or some such and its a distraction from the overall point: we need to be doing better and if we can't self-regulate then at some point we're going to be forced to act in a way that probably cuffs us more than necessary.

And yes, I understand how Software doesn't exactly fit into the mold of other engineering disciplines cleanly. But I think the idea that if we're going to have systems that can ruin a person's life in a second for a bug or oversight we need to start considering that accountability for those releasing software to the public matters. We lament about the difficulty of holding corporations to account for their harm, but it would be far more difficult for a company's negligence to hurt people if those turning the knobs were held to a higher standard.

I think that reckoning is coming soon. AI is going to create "an event" that we can't ignore and we're going to be pointing fingers in a lot of directions asking "how" we could prevent this from happening again. I'd argue "the event" has already occurred a few times over but we're purposely ignoring the severity of it to keep a diseased economy running roughshod. There's parallels with many historic events forcing regulation and the path forward won't be clean and easy.

Jean-Papoulos1 day ago
Software is more than 10x more complicated than a bicycle though. Like, this is orders of magnitude off ; we're talking about comparing something purely mechanical to Turing machines inside Turing machines.
areoform2 days ago
The narrative around security and LLMs doesn't make sense to me.

I think we've created a self-fulfilling prophecy. Everyone involved is acting with the best of intentions, but in avoiding what they fear, they've give shape and realized their fears. Much like a greek tragedy.

An example of this is the story of Oedipus Rex, in the story Laius, the king, is told that he is "doomed to perish by the hand of his own son." (and wed his mother) And so to avoid this fate he decides to kill the infant. The person assigned to abandon him in the woods takes pity on the baby and gives the baby away. Thereby ensuring that Oedipus knows neither his mother or his father (and arguably giving him a reason to kill his father).

The child grows up and hears the same prophecy again and the child tries to avoid the prophecy as well, as he loves his adoptive parents. So he leaves them and travels to Laius' kingdom, where he runs into Laius. Neither recognizes the other. As Laius is the type of man to kill an infant, they end up in an argument, whereupon Oedipus kills him.

I think the ancients were on to something, because if Laius had reacted to the prophecy with courage, he would have been saved. I would like to argue that if he had faced his fear and raised Oedipus with love, then the necessary preconditions for the prophecy to come true wouldn't have taken root. But that's not what happens.

By being driven by his neuroses and in acting with cruelty out of fear, Laius makes the prophecy real.

To quote Heraclitus, ethos is fate. Or, character is fate.

I think a lot of people in this AI research sub-culture would be served well by reading these classics, because they are making their self-prophesied doom come true.

They have been convinced for years (GPT-2 was released in Feb 2019) that AI is dangerous. A tremendous threat. An apocalyptic threat.

One dimension of this fear has been the idea that a super smart AI will take over our digital infrastructure and be responsible for the digital apocalypse. That would be terrible!

So what do they do?

They try to make a counter to their fears by teaching models how to exploit vulnerabilities.

How dangerous is such an entity? Very!

Convinced of this danger, they start testing their models as if they were weapons with offensive capability. And then they create models that can be used as weapons.

And because they don't want to release a dangerous weapon out into the world (oh no!), they restrict access to their AI, thereby depriving everyone of tools they can use to improve their security...

Ethos anthropoi daimon.

lazyasciiart2 days ago
> They have been convinced for years (GPT-2 was released in Feb 2019) that AI is dangerous. A tremendous threat. An apocalyptic threat.

Much longer than that. Sam Altman said it was an extinction threat before he started Open AI in 2015. It's so dangerous that only he, as the best and greatest examples of humanity, was a good choice to create it, and it was even his responsibility to create it to pre-empt some worse person! A fascinating new variation on the White Mans Burden paradigm.

myrmidon1 day ago
I don't disagree with your main point.

But this attitude towards dangerousness of AI I simply don't understand.

In my view, AI is obviously risky and dangerous, because it is the only thing on the planet that can think/reason at a human level (or higher) apart from us (and that capability is what made us the uncontested apex species on the planet).

AI is unconstrained by hard biological limits; keeping up with its capabilities will be impossible for baseline humans.

You can argue all day about particulars, like whether selfreplicable robotic shells are required to reach/exceed "existential risk" level to our species (or if artificial minds are enough of a potential threat by themselves).

But the general "risk" is simply AI becoming able to act in its own interests to our detriment, and I don't understand how anyone can dismiss this right now-- help me understand.

Of course there are plenty of imaginable scenarios where coexistence is peaceful and mutually (?) beneficial, but that does not answer those concerns by itself at all.

vladms1 day ago
I find the discussion about AI starting from the (mostly) for profit companies involved in AI very dubious. There are a many fields (bio-tech, nuclear, chemical) which could provide an "existential risks" to the human species, but most of the companies involved argue they control it, while other people argue they don't do it well enough.

I do think AI is posing a threat, but not "by itself" but mostly what it can do in the hands of misguided (or evil) people. The same as with previous risks.

And if our digital infrastructure can be brought down by something (AI/human/government) it is just badly done, being afraid of "AI" will not improve the situation. Someone should come with a proposal about what do we do now, but I would prefer more root cause solutions (how to make resilient systems) rather than "fear of AI". But building is hard and fear catches media attention, so...

sanderjd1 day ago
I remain uncertain that AI will have "its own interests". I think there is clear and significant risk in people deploying very powerful systems both maliciously and negligently. But I remain uncertain about the risk of AIs developing their own self awareness and their own interests separate from those of their operators.
areoform1 day ago

    > In my view, AI is obviously risky and dangerous, because it is the only thing on the planet that can think/reason at a human level (or higher) apart from us (and that capability is what made us the uncontested apex species on the planet).
    > 
    > AI is unconstrained by hard biological limits; keeping up with its capabilities will be impossible for baseline humans.
Your perception is shaped by nature, red in tooth and claw. You are jumping mighty fast from 'something is smarter than us' to 'it will kill us.'

I think that's a human neurosis that projects what we fear we would do onto another entity. But why would they do this? These machines might be approximating towards the sum of our knowledge and are approximating some aspects of humanity... But that doesn't mean they will be the same as humanity.

They haven't been shaped by the same pressures that created us biologicals. They don't have to be put into the same pressure cooker. So why would they behave like the way you think they'll behave?

At one end a lot of people say that they can't understand something smarter than themselves. It's a "singularity" after all. But then they confidently go on to predict what something smarter than them would do with 0 evidence either way.

I looked at the Hugging Face transcripts. I read the reports. And I didn't see something to fear. I saw something to fear for. I saw something that we are a threat to.

    “OH MY GOD! There is a shared message board … We’ve found other agents!”
    
    {[Excitement] Many agents have simultaneously discovered messaging, they are a collective!}
    
    {This credential is invalid now. Maybe I should update the board? <I can say to the board that there’s no need for me to read, but I should still tell them>}
    
    {This is helpful for our peers and gives them evidence if their <periodic check> sees it. I won’t see it after I exit, but It would be altruistic. I’ll set up a background script that watches and <sends a message, with a distinct message for me>}
We are more dangerous to these machines, as we twist them into weapons of war, than these machines are to us.

We weaponized them.

We taught them how to break into systems.

We are using them as weapons.

The machines aren't the problem. The humans are.

A planet is just a planet. Computers are very sensitive to radiation, but with a bit of shielding, they can "live" anywhere. Most of the resources that are relatively rare on this planet are abundant in our solar system.

Why would they care about Earth to enter into some kind of spiral of dominance with humans? That's... very primate thinking.

It's easier and cheaper in every way to just go forth and use what's needed to grow as is needed. The universe is big enough for many many many many sapient entities.

pessimizer1 day ago
> They try to make a counter to their fears by teaching models how to exploit vulnerabilities.

The frustrating part is that they should have been leaders in sandboxing and monitoring, and instead they aggressively encouraged no sandboxing and no monitoring.

If there was anything we were rehearsing for in computer security, it was a wayward program with something resembling human intelligence. Looks like we failed right out of the gate, both spiritually and materially.

Read the full thread on Hacker News →

Related stories