Hdd's and Ssd's

Hi lads, what is the best type of hard drive to farm SAFE on a PC, when it launches?

4 Likes

Too early to be sure and what future proof is for storage might also face off against surprises. At some point there will be a clearer sense of the best options for hardware but too many variables atm. Some usb c ssd might be a simple bet… but then the future could be vinyl… or tape…

eg… 100TB cartridges set to rival tapes, hard drives, DNA for data archiving domination | TechRadar

4 Likes

Best drive is probably NVMe, which have the best performance.

5 Likes

Hopefully, any hdd will do. SSD is faster than spinning disks, but the whole design is to make use of existing computers and not have folks buying specifically for Safe. Our vision is to use what folk have, which means not stressing disk drives but realising they are getting bigger and bandwidth is getting faster.

11 Likes

Hihi if anything, SAFENET will enable hyper efficiently usage of hardware. It will probably usher in dna storage @ a quicker pace and likewise Google fiber up/download “speed”. It ain’t :heart: to tell, that Nas will take over households providing storage for farmers. Hopefully these households don’t get raid in an attempt to stall the natural progression of humanity. Funny enough SAFE storage inherently serves as a counterbalance to … :exploding_head:

3 Likes

Be aware that there is a HUGE supply of FAKE drives on the market now. China is flooding the marketplace with drives that they claim have very large capacity but actually don’t and are very slow.

Most of these are unbranded, but some even fake the brand … so be very careful who you purchase from.

Search for “fake SSD on ebay”. You’ll find lots of info.

3 Likes

In Australia places like mwave.com.au and umart.com.au and ple.com.au and some others buy direct from the suppliers. Also they give good prices for the drives.

At the moment for spinning drive the 8TB seagate Barracuda drives are at a sweet price for $/TB and reliable drive.

But we have to wait since SSD prices are due to drop as the technologies filter down to the everyday SSD drives. At the moment the money for SSDs is in the special purpose SSDs and that is where the new technology is being used. (Like very large drives for servers)

3 Likes

Paypal solved this problem years ago for clueless consumers :sweat_smile:

:laughing: this is where SAFE storage does it’s balancing act, because if you hold SAFE why go through all the problem of buying or hodling a storage medium, unless your a farmer?

2 Likes

Some NAS systems combine high-performance HDDs with NVMe as buffer and in most times you get lightning fast performance with massive amounts of storage. These hybrids are probably the sweetspot of storage sharing.

1 Like

There are a number of sweet spots depending on your desired outcome.

Safe will be for space vs cost while meeting decent speed. Speed is not going to be a major factor as long as its reasonable. The lag time sending packets from node to elder to client will outweigh that saving of a few mSec

2 Likes

I was thinking to buy one of Synologies Plus series when network is live. They offer dual or quad core, nvme caching and virtual machine environment which could be used for maybe something like a docker image that spins up on the NAS and then access it’s own dedicated storage space that is served to the safe network.

Using virtualization one NAS can turn into e.g. 4 nodes to maximize chunk retrieval.

Can read more about it here, would be cool if we can get to build some images open-source that can run on these NAS so community can just dive in.

1 Like

I hear that Synology is great but I’d like to think most of us could do a bit better than that and save some cash. I really like TrueNas.

https://www.truenas.com
TrueNAS - Wikipedia.

Previously known as FreeNas and before that Nexenta. It’s FreeBSD based and uses OpenZFS as its Filesystem.

The great thing about TrueNAS is that it includes a lot of things that would normally only be found on paid for products or time consuming to setup for yourself. Things like automated backup to object storage such as AWS S3 (ok, so not really relevant for using for Safe), REST API and you can run VMs and containers on it. That is where it gets interesting for Safe.

I’ve installed it on these:- HP Proliant Microserver

Specifically the N40L. The CPU on these older models is a bit underpowered and the disk interface is slow but it’s fine for home use. HP still make Microservers but they now only have small form factor HDDs now. You can pick up the old ones with full size HDD bays quite cheaply on eBay. I’ve even run ESXi on them and had several low usage VMs running on them. Everything apart from changing disks is difficult because the motherboard has to be slid out and the cables you have to undo first are really fiddly. But they work well, are quiet and don’t draw much power.

I’ve nothing against Synology and their like. Anything that gets more people storing their own data and not losing it is good in my book but I would like to see more people increasing their knowledge and setting things up for themselves.

1 Like

TLDR - before I start my rant - watch out for those Seagate Barracuda drives!

For HDDs for NAS devices I used to buy Western Digital in 1TB, 2TB or 4TB depending on the size needed. However, I had a really bad experience with them. It turned out that the drives I had bought for a NAS system were Shingled Magnetic Recording (SMR) drives. My blood is boiling just thinking about it so I don’t want to go into all the details but basically they fit more on the platters by overlapping the tracks but that makes writing a multi-stage process. This makes writing slower. If you are only doing a small amount of writing it’s kind of ok and you won’t notice because they give them a larger cache for incoming data so it has time to write it slowly before the next writes come in or reading is needed.

However, in any kind of sustained writing like for more than a few seconds they slow to a crawl. Then obviously reads are affected as well. Whatever the system is just grinds to a halt. Sometimes to the point the system thinks the drive has failed and it’s kicked out of drive redundancy system which is what caused issues for me. Then the other drives in the system got busier of course trying to redistribute the data across the remaining members and the whole system died. It was only some test VMs on it but highly annoying.

So they are pretty much unusable for anything that people actually want to do. They do kind of have a use case - archival but you have to trickle the data in and not flood it in at high rates. But for anything else they are terrible. They aren’t even any good for use in a PC. They aren’t even suitable for security camera use.

There was a whole load of trouble caused for a lot of people finding all this out. The big problem that the manufacturers caused was when they started ‘Submarining’ the drives into existing product lines by not stating that they were SMR drives. When challenged on this they flatly denied they were doing it and said they weren’t SMR drives and that all their drives were suitable for the purpose for which they had been sold. They eventually admitted they were doing it and listed the previous and current drive models that were SMR.

See these articles:-

Before I go and try to get my blood pressure down again I’d just say - do research on any drive you are thinking of buying. What you are looking for is a statement that the drive is definitely not ‘SMR’. Preferably that it is ‘CMR’ - Conventional Magnetic Recording.

And watch out for those Barracuda because Seagate was at it as well and their Barracuda product line and 8TB+ drives were affected. That’s one of the worst things about it: they slipped them into the same product lines at the top end of the capacities trashing their previously good reputation.

2 Likes

Might Safe in fact be a suitable application for SMR drives?

We’d need to look at the write rates for chunks etc in different node activities but in general it might well be that we’re rates are relatively low on average because of the way chunks are distributed randomly across many nodes.

Do you have any figures for write limitations for SMR drives, just ball park would be useful?

4 Likes

That’s a valid question and worth pursuing. I’ll look into it. I still have the drives and I’ll do some testing when I have some time.

A warning sign that a drive is SMR is if you see it has a cache of 256MB rather than a more normal 64MB. So that could be the ballpark figure you want. When that cache becomes full there will be a slowdown. So the real question will be how quickly the drive can write to actual disk? We know it’s very slow but it needs testing.

They could well be fine for Safe now I think about it. It is sustained writes they can’t handle. If they just get a small amount of writes they have decent performance because they have a larger than normal write cache. Also, when the writes go to disk they write to a non-shingled area and later on write to a shingled area. What size those areas are and the rate at which they can de-stage to them will determine how many writes they can take over what period.

If that write rate over a time they can supply is greater than what a Safe node demands they could well be fine.

But they are totally unsuitable for use in NAS devices because not only do they tend to have periods of intensive writing (doing a backup to them, storing a new video, etc) they also do consistency checking or a ‘scrub’ of the data such as on TrueNAS. The biggest issue though is if RAID has to be rebuilt. The drives just can’t keep up with that. Never use them in any kind of RAID or RAIN setup or there will be tears.

I’d be delighted if these drives do have a use case in Safe because they are cheaper per TB and they use less power and emit less heat for their size due to fewer platters to spin. Otherwise they were a totally flawed concept outside archival and only when data isn’t coming in in big batches which isn’t likely. Western Digital made very large Object Storage systems (they’ve since sold the business to Quantum) and even they didn’t use their own shingled drives in them! That says a lot. They used 12TB and 14TB Helium filled drives that cost a pretty penny 3 years ago.

Check out this article and discussion from the Storj forum I’ve just found. They were warning people about the drives pointing out that with all the writing to Storj at the time traffic was constant and heavy.

I wonder if when Safe is live traffic could sometimes be heavy for individual nodes? Also, if when a node is joining a Section or being promoted there might be more traffic for it than these SMR drives can handle? One to watch!

2 Likes

Thank you for the information, I had not known. Of course these drives are not NAS and really shouldn’t be used for them with NAS that is reasonably active.

Good thing SAFE is not a local RAID and/or NAS system, the write to the drives is random and occasional with absolute max write rate on the order of your internet connection. Typically will be a lot less, quite a lot less since SAFE is more of a write once and read often. (Once every time node is relocates)

I have Ironwolf pros for my NAS

When testing the Barracuda drives I received, I did not notice the issue, I will keep an eye open when backing up my data onto them now that you have warned me.

They should be just as fast for append only writes if you are writing in sequence. Might be a future improvement to the safenet, but would require a custom safenet SMR files system of sorts.

2 Likes

Btrfs does already support smr disks. Unfortunately only host managed smr, that is disks that do not pretend to be normal disks (drive managed), but actually give the system the knowledge about the disks setup and the system has to handle the zoning/smr by itself. The stupid part: you can’t get host managed smr disks from the retail market, there are NONE!

1 Like