And they had zero notice?
That’s kind of how bankruptcy works. You can’t give notice that you’re going to be bankrupt. The moment you’d give notice of bankruptcy, everything would collapse anyway as lenders call in debts and customers cancel orders that are now in jeopardy.
Other statuses that exist are examinership (an outside appointee trying to save the company by any means) and receivership (probably the closest to what you’d like - the company is entrusted to a third party to try to pay its debts before it is wound up, usually trying to determine which debts to pay for the greater good, and which ones to default on).
The cloud is just somebody else’s computer.
Hehe. Was just about to post this. Word for word
I know Wikimedia is getting deserved shit for what they’re doing now, but allowing anyone to download the entire shebang means it’s far less likely it will be lost in the future.
If PBS wasn’t forced to make money they could have done the same with their content. The only reason they needed to control it was because they needed to sell it.
How fucking lazy do you have to be to use cloud storage?
how fucking entitled do you have to be to assume your world view is the only view?
so invest in friggin local storage, wtf
Jesus Christ, what happened to P2P? Just have the files in constant circulation through the world’s computers. And 50TB, while a lot, isn’t beyond the reach of a well-heeled private individual to host on a NAS.
50TB is not a lot, it’s actually a capacity that some homelabers and data hoarders have or surpass.
It’s a lot if you were trying to source the hardware with today’s process, but 5 years ago that would be nothing to an organisation.
It’s a 4-digit number, even today, to build a 50tb local backup box. Absolutely nothing to a company. They were simply lazy and didn’t do the bare minimum.
Or they were sold a bill of goods by a cloud vendor.
I’ll store PBS’s content on my Plex server if they want. I’m sure I’ve got 50Tb free
This is PBS St Louis. So get ready to store hundreds of hours of this:
https://youtube.com/playlist?list=PLOBToc54vT5C1tdWJuXBFFD1eh7DCI4CR
I like the music but you’re right, I probably won’t watch 50Tb of talkback tv
This looks like this is isolated to KETC’s locally produced shows for Missouri. They appear to be a smaller affiliate and they don’t make a ton of stuff that contributes to national and international broadcasts.
Since they have a smaller audience I imagine it’s going to be harder for KETC to find community-created backups of their stuff.
I also imagine that if Missouri gave more of a shit about public broadcasting, this affiliate would be operating with better funded / more experienced IT staff. These little affiliates really rely on a lot of volunteers and green IT folks who do the work for the experience.
I do volunteer work for nonprofits. Its amazing that normal people think we are swimming in cash… newsflash, we are not.
The IT budget is nothing so alot of what i do is setup open source stuff to meet needs…think libreoffice erc.
External storage for large backups is not on the table. And no one wants the reaponsibilty of shuffling external backup drives to and from.
And IT staff is often a lot of young folks who are building their resume and make really dumb rookie decisions. I was once that rookie.
All the people in the thread dissing the station and suggesting all they needed is “X many Y terabyte drives for $2.50 each and give them to people to take off site and store and rotate somewhere and all this wouldn’t be a drama”… Those people have yet to discover that manual processes using fallible humans does not work long term.
The station’s IT staff no doubt looked at all of this and decided that off-site backups via a third party was the simplest and most cost effective route. I have no doubt the middle-company advertised that all the actual storage was in Backblaze, a quite reputable storage company. The only thing that has bitten them in the ass is that the middle-company has folded unexpectedly, requiring them to scoot to court to get Backblaze to give them access to their data.
After all of this, it’s quite possible that IT will now get enough resources to do the backup via Backblaze directly. If Backblaze wants to put a bit of PR spin on it, they’ll task a few engineers to set this up for them. And if Backblaze was smart, they’d go looking for alllll the other little PBS affiliates in the same boat and offer them the same setup.
50TB isn’t even that expensive on a HDD! Couple hundred to a thousand bucks! How much were they paying for that much storage?
The first time I heard about cloud storage, I wondered why anyone would store their stuff with a third party, for reasons EXACTLY like this. This is the exact scenario I thought of.
And while we’re talking about this specific case, presumably all the other customers of that cloud storage company list all their stuff, too.
Using cloud is not the problem here and never has been an issue from a technical design standpoint, it’s having the data only with one cloud provider that allows this to happen.
The only customers of that cloud provider that lost their stuff are the customers that would have lost their stuff using any other method they’d implement, because the only way you can lose your data this way is through poor design.
Someone just tell me that there are backups of most of this properly archived somewhere. Just tell me that, I’m not asking if it’s true, I just need someone to say that.
Likely available on a torrent.
Does the local St. Louis affiliate produce anything that lots of people would want to seed? They look like a smaller affiliate compared to the WGBH and WNETs out there.
I imagine KETC is a really small shop that mostly broadcasts stuff others have made, plus hyper-local content that isn’t expensive to make.
The data is all there, the vendor they were using to manage the storage just went OOB, and they’re not yet being allowed to retrieve it from the vendor’s storage vendor, Iron Mountain:
The lawsuit filed in Denver District Court alleges that the station’s cloud-storage vendor, Open Source Storage, abruptly cut off access to Nine PBS’ data earlier this year without warning. It states OSS, which had a separate relationship with Iron Mountain to provide data storage, went “defunct,” leaving Nine PBS’ archives in a data center operated by Iron Mountain.
Open Source Storage
Anyone heard of this company before? Their name is impossible to search for!
Thanks. This is the latest archived copy: https://web.archive.org/web/20250328163113/https://www.osstorage.com/
End-to-Endless Digital Storage Solutions
Endless? lmao
Iron Mountain is obviously aware of the situation and probably doesn’t want the bad press of deleting it. PBS just needs the court order that makes them hand the data over. Because PBS isn’t iron mountains client, they can’t hand them the data without a court order.
Redundancy would have made this a non issue. Even just local cold storage.
50 TB isn’t even that much space, certainly not for a broadcaster. No reason not to have a local copy. 50 PB, I could see a small PBS affiliate not having the resources to manage locally, but you can put 50 TB in consumer-grade NAS.
That’s less storage than my Plex storage. PBS could and should definitely do better.
More specifically, this PBS affiliate (Nine PBS St Louis). It wasn’t PBS proper. This only impacted local programming archives. But that’s still a loss for the local community.
PBS likely does much better than this. Heck, a lot of their stuff is backed up in the Library of Congress.
That’s fair. The point still stands that they should do better.
Agreed. Although sometimes these smaller affiliates don’t have super-seasoned IT talent. The jobs for the department heads at these organizations often make what an intern or college grad would make in the private sector at an unremarkable mid-size tech company.
You often get people making questionable decisions simply because they don’t have seasoned IT people in the building.
All in all, this was dumb, but it also feels like a story I’ve seen before. The nonprofit media isn’t funded, and the cracks are showing.
When I started at my last job, I was fresh out of college, only Best Buy Geek Squad experience, and I was the sole IT guy at the company. All “backups” were shadow copies and VM snapshots. Shadow copies are great for simple accidental deletions, and the VMs would work as a backup 90% of the time, but it was not a valid backup solution. When I left we had onsite backups to hard drives, cloud backup in Azure blob storage, and remote replication geographically distant.
It still could have been better, but it was something. Also, we worked on government contracts and stored confidential and ITAR controlled military and government drawings and data. It was one of those things that I’m absolutely shocked that in my 9+ years there that we never had a major breach or lost any data.
Either way, like under $2000 would have bought them the storage necessary to keep these backups on-site. They still would’ve needed another backup solution for redundancy purposes, but this wouldn’t be the huge issue it seems to be now, where the data is sort of in limbo.
You can put this on 2 HDDs nowadays. This is pathetic.
50T isn’t even that much data anymore. You’re talking about maybe $2k in hard drives.
In the pre-AI days maybe, today 50TB is expensive due to increasing prices of all computer gear.
today 50TB is expensive
I don’t believe you can go back more than five years and find per-TB rates under the current rate. Ten years is straight out. Twenty years is laughable.
Meanwhile, PBS has an annual operating budget of around $340M. They can afford to manage their own data, at the scale of tens of terrabytes. And I guarantee you that whatever they’re paying Iron Mountain to host the data is multiples the cost of self-hosting.
Refurbished Exos X22 drives with 24GB Capacity cost around 650€ these days, those 2k are the high end even nowadays.
Refurbished
If you’re a multi-billion dollar broadcast parent company doing long term storage for a 70-year archive catalog, you can spring for new. Maybe even go Big Fancy and get some RAID drives and tape backups, for extra redundancy.
Piracy is media preservation
…
Iirc is “The Owl House”
I plead the fifth on that. But how come no backup on this. I am pretty sure they could got at least 50 people to get 1 tb hard drives so their data wouldn’t go this route. And probably more since PBS shaped whole generations of kids and adults into better people.
15x8TB drives gives you about 72TB in raidz2, for about £4k in disks, add a server for say £2k
So by not spending £6k they lost everything, they spent $19million on IT in 2025
Someone fucked up
A tiny PBS office in St Louis has a $19 million/year IT budget? I really doubt that.
I may have taken the whole PBS accounts, I’m not from the US
you forgot the $10K in labour to handle the backups
changes everything on that $19M budget!
That’s budget for the whole network, this happened to a small local station.
Heck its pbs. They should have publically archived it so anyone could keep a copy. At least the old stuff.
According to the article, the data is not clearly lost. It’s that their provider, OSS, had a subcontract with Iron Mountain to host the data storage. When OSS abruptly went defunct, PBS had no direct legal path to access the data in Iron Mountain, and no assurance Iron Mountain would preserve it. It filed the lawsuit to make sure the data was preserved and affirm a legal basis to recover the data.
So the headline’s use of “loses access” is accurate but a little misleading. That said, it is a great argument for reliable data preservation.
also why you should have local and remote backups both.
Gotta follow the 3-2-1 rule
Iron Mountain is aware of the situation so ideally they’ve flagged it to avoid the bad press of deleting it.
Well, yes, but consider that others using the OSS service may not have the resources of PBS to file suit and litigate the issue, and they may well lose their data forever.
The whole company is sketchy
https://web.archive.org/web/20250318131517/https://www.osstorage.com/cloudPricing
There is nothing open source, their address is the iron mountain denver datacenter address
That page looks like it was directly ripped from https://wasabi.com/pricing, including the “Ball appliance” which appears to be a Wasabi-only product. This reeks of “I know a guy who does storage” and it’s just a few open source products bolted to another storage API with upcharging and a corporate website
Good argument to not use the service.
