“I store my records on vinyl. You’ve probably never heard of them.”
lol guys I think this is a shit post
i mean…
If you’re running a pcie nvme ssd, one of the modern ones, and you’re doing a SHIT ton of reads, like threadripper level amount of reads, i guess “overheating” isn’t unexpected? Shouldn’t do much other than slow down the SSD though?
dumbass probably loaded them into memory, and OOM’d, and thought it was the drive.
NVMEs will throttle if they are inadequately cooled. Pretty much only folks who are new at buiding computers and don’t adequately cool their NVMEs experience this.
Either way it shows what a clown show this is. I’m pretty sure they’re doing some babuki theatre to distract us and then if the scheme being done here is found, they’ll point fingers at the dipshit kid and he’ll be punished for whatever the con is.
Either way the billionaires will be better off for it, barring judicial, congressional, or military revolt; which is looking less likely by the day.
Bunch of traitors forgot their oath is to the constitution. The constitution IS the soul of the country and is to be respected. The country was founded by the constitution and it defines the system to be decentralized power giving control to the populace through three different branches of government. Not some false idol king or an evil oligarchy propped up by foreign and domestic robber barrons
Out of memory/overheating in 60k rows? I’ve had a few multi-million row databases that could fit into a few gigs of memory, and most modern machines have that much in RAM. A 60k query that overheats the machine might only happen if you’re doing something weird with joins.
Plus a lot of reads is nothing really, for basically all databases, unless you’re doing an unsmart thing with how you’re reading it (like scanning the whole database over and over). If you’re not processing the data, it’d be I/O bottlenecked.
if they wrote good code yeah, evidently they didn’t write good code if they’re struggling to process 60k lines of a database lmao.
They must either be O(n^10) complexity or something retarded like that for this to be the case. I wouldn’t put it past them.
Plus a lot of reads is nothing really, for basically all databases, unless you’re doing an unsmart thing with how you’re reading it (like scanning the whole database over and over). If you’re not processing the data, it’d be I/O bottlenecked.
again, i’m assuming they aren’t very smart, since this is an issue in the first place.
I used to perform data analysis of robotics firmware logs which would generate several million log lines per hour and that was my second job out of college.
I don’t know how you fuck up 60k lines that bad. Is he nesting 150 for loops and loading a copy of the data set in each one while mining crypto??
I would love to see the output of EXPLAIN on his query to see what he’s doing.
Substring searches in unindexed large string columns or cartesian explosion caused by shitty joins would be my initial guess.
Largely ignorant, but data-curious person here.
…what?
If there’s something you want to search by in a database, you should index it.
Indexing will create an ordered data structure that will allow much faster queries. If you were looking for the username gazter in an unindexed column, it would have to check literally every username entry. In a table of 1000000 entries it would check 1000000 times.
In an indexed column it might do something like ask to be pointed to every name beginning with “g”, then of those ask to be pointed to every name with the second letter “a” and so on. It would find out where in the database gazter is by checking only six times.
Substring matching is much more computationally difficult as it has to pull out each potentially matching value and run it through a function that checks if gazter exists somewhere in that value. Basically if you find yourself doing it you need to come up with a better plan.
Cartesian explosion would be when your query ends up doing a shit load of redundant work. Like if the query to load this thread were to look up all the posters here, get all their posts, get the threads from those posts and filter on the thread id.
That’s very clear, thanks.
I’m guessing you’d have to search the database to make the index, right? To search for ‘gazter’ you’d have had to go over the whole dataset and assigned each entry with a starting letter value, and so on?
When it comes to searching the database, the index will have already been created. When you create an index, it might take a while as the database engine reads all the data and creates a structure to shadow it. Each engine is probably different and I don’t know if any work exactly like that, but it’s an intuitive way to understand the basics of how B-trees work. You don’t really need to think much about how it works, just that if you want to use a column as a filter, you want to index it.
However, when you’re thinking about the structure of a database it’s a good idea to think what you’ll want to do with it before hand and how you’ll structure queries. Sometimes searching columns without an index is unavoidable and then you’ve got to come up with other tricks to speed up your search. Like your doctor might find you (i’m presuming gaz is sort for gary and/or gareth here) with a query like
SELECT * FROM patients WHERE birthdate = "01-01-1980" AND firstname LIKE "gar%"The db engine will first filter by birthdate which will massively reduce the amount of times it has to do the more intensive LIKE operation.
Storing large volumes of a text in a database column without optimization, then searching for small strings within it. It causes the database to basically search character by character to find a match by reading everything from disk. If you use indexes the database can do a lot of really incredible optimization to make finding values mich faster, and honestly string searching is better suited to a non-relational DB engine (which is why search engines don’t use relational DBs).
Cartesian explosion is where you join related data together in a way that causes your result set to be wayyyy bigger than you expect. For example if you try to search through blog posts, but then also decide to bring in comments to search, then bring in the authors of those comments and all their comments from other posts. Result sets start to grow exponentially in that way, so maybe if you only search a few thousand blog posts you might be searching through millions of records because you designed your queries poorly.
I didn’t know hard drive overheating was a thing. Should I be worried that my 5 year old hard drive is about to overheat. I mean is this actually a floppy disk or something?
When an HDD works continuously it can heat up to above 60 °C if proper air circulation is not allowed, which can cause a very premature failure. In fact, it should be kept under 40 °C to achieve the intended lifespan. Unfortunately, PC cases are usually not great at removing heat from the HDD by default.
As for your drive, it most likely has a temperature sensor so it can be displayed by various utilities.
I have a 12v fan running at 5v spitting air on my hdds, and that’s enough for them to go from 55°C to 29°C, lol.
I did that too, the tiny fan is pretty much silent at 5V. The HDD has so much surface area it only needs a little air circulation.
Removed by mod
it is, in the select event that your platter bearing fails, in which case it would be very, very obvious.
So you’ve had this happen?
no. but generally spinning things that spin at several thousands of RPM that are spinning on a bearing, that no longer have a bearing usually sort of uh, tend to be VERY noisy.
Ok, but do you know anyone this has happened to? I don’t and I have some pretty old drives I still use. I tend to just buy more. Also most drives these days are solid state aren’t they? This just feels like a low probability event to me. Overheating RAM or the CPU or GPU, sure, but hard drive?
not personally, i may have seen a video or two of it happening, but it’s hard to tell whether the head is dragging against the platter, or it’s the bearing, either one of those makes horrendous noise.
If you’re worried about it happening on a drive you own, you should copy that data somewhere else as a backup, ideally sooner rather than later. If you’re curious about the health of the drive you do stuff like SMART tests as well.
Yeah, most drives are solid state now, unless you’re buying hdds for archival purposes, still cheaper and denser in most cases. It’s a low probability failure, until the drive meets EOL, in which case it’s a mechanical wear part, either the motor or the bearing fails. One of them will fail first, probably the bearing.
The bearing failing would likely result in the HDD overheating as a result. Assuming the platter still spins, but that’s the only scenario i can think of where that would happen, unless you dump a very specific amount of continuous current into the read arm coils. That might also cause it, but it’s not likely at all.
An ssd “overheating” is more likely, but it shouldn’t cause too many issues, maybe premature degradation over long term use, and slowing of read/write speeds, or in some cases, an improvement, but other than that it should be business as normal. You would have to hit it with like a heat gun, to get a hardware failure or something like that.
I’m not actually worried about my drives. In fact that was kind of the point. I was kidding around because this excuse that the hard drive overheated sounds a little like a car running out of blinker fluid.
Removed by mod
I had it happen to a random hard drive I bought, old bastard found in a bin at a local thrift store. Anyways had a big dent in it and while it started up, even booting into windows XP it cooked itself made a screeching noise and was hot to the touch. Don’t think I need to explain that the big dent was probably the source of the overheating, anyways had to use an oven mitt to relocate it to a metal bucket of water where it boiled for a minute.
So the story is plausible then?
my hard drive overheated
So, this means they either have a local copy on disk of whatever database they’re querying, or they’re dumping a remote db to disk at some point before/during/after their query, right?
Either way, I have just one question - why?
Have you ever heard of case of overheating hard drives within the last decade?
I don’t think I’ve seen a brand new computer in the past decade that even had a mechanical hard drive at all unless it was purpose-built for storing multiple terabytes, and 60K rows wouldn’t even take multiple gigabytes.
Reminds me of those 90s ads about hackers making your pc explode.
Musk gonna roll up in a wheelchair, “the attempt on my life has left me ketamine addicted and all knowing and powerful.”
You’ve got it all wrong, in traditional computer terminology the “hard drive” is the box that sits under the desk that collects cat fluff and cigarette tar.
/s …?
Plus, 60k is nothing. One of our customers had a database that was over 3M records before it got some maintenance. No issue with overheating lol
I run queries throughout the day that can return 8 million+ rows easily. Granted, it takes few minutes to run, but it has never caused a single issue with overheating even on slim pc’s.
This makes no fucking sense. 60k rows would return in a flash even on shitty hardware. And if it taxes anything, it’s gonna be the ram or cpu- not the hard drive.
In my experience, the only time that I’ve taxed a drive when doing a database query is either when dumping it, or with SQLite’s vacuum, which copies the whole thing.
For a pretty simple search like OP seems to be doing, the indices should have taken care of basically all the heavy lifting.
Pretty sure I run updates or inserts that count over 60k fairly often. No overheats. Select queries sometimes way higher.
I literally work with ~750,000 line exports on the daily on my little Lenovo workbook. It gets a little cranky, especially if I have a few of those big ones open, but I have yet to witness my hard drive melting down over it. I’m not doing anything special, and I have the exact same business-economy tier setup 95% of our business uses. While I’m doing this, that little champion is also driving 4 large monitors because I’m actual scum like that. Still no hardware meltdowns after 3 years, but I’ll admit the cat likes how warm it gets.
750k lines is just for the branch specific item preferences table for one of our smaller business streams, too - FORGET what our sales record tables would look like, let alone the whole database! And when we’re talking about the entirety of the social security database, which should contain at least one line each in a table somewhere for most of the hundreds of millions of people currently living in the US, PLUS any historical records for dead people??
Your hard drive melting after 60k lines, plus the attitude that 60k lines is a lot for a major database, speaks to GLARING IT incompetence.
My one question would be “How?”
What the hell are you doing that your hard drives are overheating? How do you even know it’s overheating as I’m like 90% certain hard drives (except NVMe if we’re being liberal with the meaning of hard drive) don’t even have temperature sensors?
The only conclusion I can come to is that everything he’s saying is just bullshit.
They have temp sensors. But have never heard of a overheating drive.
Hard drives do get hot and need some cooling but not at 60k rows. Its either made up or their computer case is made of thermal cladding
Imo if they can’t max out their harddrive for at least 24 hours without it breaking, their computer was already broken. They just didn’t know it yet.
Any reasonable SSD would just throttle if it was getting too hot, and I’ve never heard of a HDD overheating on its own, only if there’s some external heat sources, like running it in a 60°C room
You could query 60,000 rows on a low tier smart phone. Makes no sense at all.
Can we think of any device someone might have that would struggle with 60k? Certainly an ESP32 chip could handle it fine, so most IoT devices would work…
Unless the database was designed by someone who only knows of data as that robot from Star Trek, most would be absolutely fine with 60k rows. I wouldn’t be surprised if the machine they’re using caches that much in RAM alone.
Right? There’s no part of that xeet that makes any real sense coming from a “data engineer.”
Terrifying, really.
Even if it was local, a raspberry pi can handle a query that size.
Edit - honestly, it reeks of a knowledge level that calls the entire PC a “hard drive”.
Unless they actually mean the hard drive, and not the computer. I’ve definitely had a cheap enclosure overheat and drop out on me before when trying to seek the drive a bunch, although it’s more likely the enclosure’s own electronics overheating. Unless their query was rubbish, a simple database scan/search like that should be fast, and not demanding in the slightest. Doubly so if it’s dedicated, and not using some embedded thing like SQLite. A few dozen thousand queries should be basically nothing.
Yeah, no matter what way you disorganize 60,000 rows, the data is still going to read into memory once.
I’d much sooner assume that they’re just fucking stupid and talking out of their ass tbh.
Same as Elon when he confidently told off engineers during his takeover of Twitter or gestures broadly at the Mr, Dunning Kruger himself
Wonder if it’s an SQL DB
Elon probably hired confident right wingers whose parents bought and paid their way through prestigious schools rather. If he hired anyone truly skilled and knowledgeable, they’d call him out on his bullshit. So the people gutting government programs and passing around private data like candy are just confidently incorrect
Why? Because they feel the need to have local copies of sensitive financial information because… You know… They are computer security experts.
Or they’re doing it on a Diamondmax 9.
I will now attempt a full comment … brace your hard drives
Bro seriosuly fuck off my phone is overheating now. Thanks
The trick is to take the hard drive out of the EVA foam.
Instructions unclear, dick stuck in GPU.
I cannot believe these people make more than me lol.
They make nothing. They’re compensated for destroying things, and considering it’s musk, they’re likely given relatively little money in return for their time.
Even if the only thing you do all day is sit on the toilet and yell at the Internet, you’re already a bigger net positive on society.
Most likely compensation is the promise of being part of privatization of whatever the fuck they’re destroying.
They’re dumb kids. They probably see Musk as some kind of god. He gave them some line like “Hey, you guys wanna save the United States?” and they jumped at the chance.
You get to be a piece of history! Whheeeeeee
As a former dumb kid this tracks
We’re going to see all of these clowns in front of a grand jury one day.
I would be absolutely shocked if we had anything approaching justice for what this administration is doing.
We barely got anything for that whole ass insurrection attempt.
The reason he recruited a bunch of incompetent college kids is because nobody with any experience or wisdom would touch this rolling crime wave with a 10 foot pole
What is this, a table for ants? Because that’s the average number of ants in an ant colony and it’s nowhere near an impressive amount of rows to be doing any sort of processing on. It wouldn’t be an impressive amount of rows if your rig was an i386DX-33 running off a 5” floppy.
Exactly, 60k rows is negligible enough in most cases that you can just treat it as free unless you’re doing a cross join on it or something, unless he’s doing something like using an unordered text file as his database with no ram or cache
Buddy’s probably running code he got from GitHub Copilot that is used to do a visualization of a bubble sort for learning purposes.
Even then, I can crank through orders of magnitude more data on my craptop
I’m a data engineer that processes 2 billion row 3000 column datasets every day, and I open shit in Excel with more than 60k rows. What the hell is this chick talking about?
Seems like a good excuse to someone who doesn’t know what they’re doing and needs an excuse because why they haven’t completed it yet?
The whole post is complete bs in multiple ways. So weird.
It sounds like Hollywood tech lingo. Like when you’re watching a movie or a TV show and the designated techy character starts just saying computer words that make no actual sense in the real world, but I guess in CSI: Idiottown the hard drives have severe overheating issues.
If you work for a boss that fundamentally misunderstands what you are doing, then misleading them into thinking you’re ‘hard at work, making decisions with consequences’ is the theatre you put up to keep the cash flowing.
It’s one of the fundamental flows of autocracy, people try and represent what you want them to
Some interesting facts about excel I learned the hard way.
- It only supports about a million or so rows
- It completely screws up numbers if the column is a number and the number is over 15 digits long.
Not really related to what you said, but I’m still sore about the bad data import that caused me days of work to clean up.
The row limitation seems, to me, like an actually-good thing. Excel is for data where you might conceivably scroll up and down looking at it and 1M is definitely beyond the ability of a human even to just skim looking for something different.
An older version of Excel could only handle 64k rows and I had a client who wanted large amounts of data in Excel format. “Oh sorry, it’s a Microsoft limitation,” I was thrilled to say. “I have no choice but to give you a useful summarization of the data instead of 800k rows (each 1000 columns wide) of raw data.”
Some time ago, I heard a story of CS and Econ professors having lunch together. The Econ professor was excited that Excel was going to release a version that blew out the 64k row limit. The CS professor nearly choked on his lunch.
Dependence on Excel has definitely caused bad papers to be published in the Econ space, and has had real world consequences. There was a paper years ago that stated that once a country’s debt gets above 120% of GDP, its economy goes into a death spiral. It was passed around as established fact by the sorts of politicians who justify austerity. Problem was, nobody could reproduce the results. Then an Econ undergrad asked the original author for their Excel spreadsheet, and they found a coding error in the formulas. Once corrected, the conclusion disappeared.
It completely screws up numbers if the column is a number and the number is over 15 digits long.
I work in insurance in Brazil, by standards of our regulatory body, claims numbers must be a string of 20 numbers (zfill(20) if needed). You can’t imagine the amount of times excel had fucked me up rounding down the claim numbers, this is one of the first things I teach to my interns and juniors when they’re working with the claims databases.
Could you import the column in as text to preserve it?
This sounds like trying to do stuff in Excel? The computer isn’t overheating but the amount of memory needed is very high which would make it run poorly. They might interpret that as overheating?
Yeah, everyone commenting about being able to handle billions of rows easily, which obviously very true if you are worming with sql or similar.
But this is probably some finance kid, investment banker analyst, and only knows how to use Excel.
60,000 rows in excel with formulas, if not done efficiently, can for sure make you computer a little toasty.
Could be a gen 5 nvme drive without adequate cooling. Them bastards can run hot. Especially the early gen 5 drives.
But with only 60k rows would one even have enough time to overheat?
It also makes sense if they are on calling the entire computer “the hard drive” like grandma and the fans kicked on.
Maybe he left his shed lizard skin on top of the hard drive that caused the overheating?
Is this a real post? I can’t seemed to find it on that website “X, formerly known as Twitter.”
Yep, it’s real. It was a reply, so maybe that’s why you’re having trouble finding it. Here’s a nitter link to it.
Thanks. I’ve legit never understood how to navigate Twitter. Both Twitter and Snapchat gave me a good dose of what I imagine my grandmother feels when faced with operating a desktop computer…
some poeple have linked the discussion in other comment threads
60k lol.
I regularly work with data in the 16tb range and weirdly my computer is fine. Git gud, doge scrubs.
