Torrents -> Usenet: What should I know?

[vilane]

hello all,

one of my coworkers has been telling me that I would be much better off going Usenet instead of my VPN Torrent approach to scraping shows and movies. I wanted to get some opinions and maybe even some war stories/experiences with anyone transitioning from using torrents to usenet.

I intend to use it in conjunction with Sickrage and Couchpotato that I already have configured using torrents through Transmission. I am toying with adding Headphones, but not sure how much music/genres/bands I would actually want to listen to is out there for me to grab. All of this appears to be pretty simple to toggle/swap between torrents and nzb so I am not really worried about it being an intensive shift. I suppose my main hesitation is finding a good indexer and service and then whether or not the potential additional cost per month over my VPN cost now is worth the gains/throughput.

Thanks in advance for the insight.

 

[danb35]

For an indexer, I use usenet-crawler.com, which is free and seems to work pretty well (at least, well enough that I haven't felt the need to find a better one). As a very rough comparison, I'd say that there are probably more torrents out there than nzbs, but having found an nzb, there's a much better chance of being able to actually download what you're looking for. I pay $10/mo for usenetserver.com, and they have no trouble saturating my 50 Mbit connection, though I usually throttle that down to leave some bandwidth for other uses.

 

[TheDubiousDubber]

Switching to usenet years ago was one of the best decisions I made. I have an unlimited plan through astraweb for $11/mo, which isn't that much more expensive than my VPN service through PIA. I have been using nzbs.org as an indexer as I was lucky enough to come across open registration. But, there are plenty of searches and indexers out there, if you make the switch I'm sure you won't regret it. Plus a lot hosts have limited plans, so you can try it out and if you don't like it then you can always go back to torrenting.

I should also note. Astraweb has no problem saturating my 105Mbps connection. I didn't even realize I had been bumped from 50 to 105 by my ISP until my downloads shot up from around 7Mbps to 13+Mbps.

 

[vilane]

So as a follow up to this and the comments and discussions with colleagues, I took the plunge.

10.4 mbps download speed and roughly a day later, I had successfully snatched almost all of my backlog search queue in SickRage and Couchpotato.

The cost is a little more per month than I was spending on the VPN, but for that speed, availability, and reliability I will gladly be paying this price going forward. Unless you are only grabbing bleeding edge torrent content that is hot and just released, you will be often hard pressed to get seeders in a swarm that will allow you to churn through snatching content that quickly. Not to mention I couldn't even shake a stick at the throughput I get on usenet versus torrents. It is almost doubled what I was getting on a good day with torrents with all cylinders firing.

tl;dr Usenet may cost a little more, but there are so many more benefits to using it than torrents.

 

[titan_rw]

I've been downloading off of usenet since about 2000. Maybe 2001. Way back when 7 days retention was unbelievable.

I only use torrents when I absolutely have to, ie I can't find it on news. And it's a private tracker so there's virtually zero chance of dmca problems.

 

[jgreco]

I've been downloading off of usenet since about 2000. Maybe 2001. Way back when 7 days retention was unbelievable.

And you know, the funny thing, there's a tie-in here. Back around 2000 I picked up Newshosting as a client, and helped them make some aggressive technical moves towards a distributed spool architecture. This included moving away from direct-attach storage arrays and eventually we built one of the very earliest 24-drive in 4U servers that was based, not on SCSI, but on cheap commodity SATA disks. And it was all network-based, so each front end server could talk to several (or MANY) spool servers. This poured gas on an already simmering retention war between several of the larger providers, some of whom (like UseNetServer) were highly invested in extremely expensive FC SAN gear.

There are several interesting things: first, was that it was actually the conversion from ATA to SATA that made inexpensive large-scale arrays possible and practical. Prior to that, ATA cabling and controllers had been extremely limiting. Second, the price points were so horribly off for high-quality SCSI or FC gear that you could buy three or four times the amount of storage with cheap SATA arrays, but the storage wasn't protected by a hardware RAID controller, so we did it in software, storing redundant copies, kind of like ZFS.

Some of you will have probably noticed that I'm an advocate of using inexpensive (not the nearline "enterprise", but rather desktop or NAS-class) SATA disks where possible, and a lot of that is based on the observation that one can buy a LOT more redundancy for the same price using less expensive disks, which are usually the major cost component of a larger storage array. I firmly believe that you can get better reliability and more capacity at a lower cost that way, comments borne out by observation of the deployment of thousands of disks used to store petabytes of data.

Newshosting had a lot of fun with this because for a long while they maintained higher retention than they advertised (and higher than anyone else in the business) and any time the competition upped retention, Newshosting just turned up some configuration settings to go a few days better. Cheap storage wins almost every time.

 

[Fuganater]

I've only used torrents for years but this sounds interesting so I might give it a try. From what I have read in 5 minutes is that it is way faster than torrents.

 

[cyberjock]

And you know, the funny thing, there's a tie-in here. Back around 2000 I picked up Newshosting as a client, and helped them make some aggressive technical moves towards a distributed spool architecture. This included moving away from direct-attach storage arrays and eventually we built one of the very earliest 24-drive in 4U servers that was based, not on SCSI, but on cheap commodity SATA disks. And it was all network-based, so each front end server could talk to several (or MANY) spool servers. This poured gas on an already simmering retention war between several of the larger providers, some of whom (like UseNetServer) were highly invested in extremely expensive FC SAN gear.

There are several interesting things: first, was that it was actually the conversion from ATA to SATA that made inexpensive large-scale arrays possible and practical. Prior to that, ATA cabling and controllers had been extremely limiting. Second, the price points were so horribly off for high-quality SCSI or FC gear that you could buy three or four times the amount of storage with cheap SATA arrays, but the storage wasn't protected by a hardware RAID controller, so we did it in software, storing redundant copies, kind of like ZFS.

Some of you will have probably noticed that I'm an advocate of using inexpensive (not the nearline "enterprise", but rather desktop or NAS-class) SATA disks where possible, and a lot of that is based on the observation that one can buy a LOT more redundancy for the same price using less expensive disks, which are usually the major cost component of a larger storage array. I firmly believe that you can get better reliability and more capacity at a lower cost that way, comments borne out by observation of the deployment of thousands of disks used to store petabytes of data.

Newshosting had a lot of fun with this because for a long while they maintained higher retention than they advertised (and higher than anyone else in the business) and any time the competition upped retention, Newshosting just turned up some configuration settings to go a few days better. Cheap storage wins almost every time.

The first SATA spec was ratified in 2003. No clue when the first chipsets were made, but you definitely weren't using SATA in 2000/2001. ;)

 

[jgreco]

The first SATA spec was ratified in 2003. No clue when the first chipsets were made, but you definitely weren't using SATA in 2000/2001. ;)

Right, we were experimenting with ATA first. Perhaps you missed the part where I said "the conversion from ATA to SATA made [...] possible and practical." But even before that, which is what allowed a true explosion in density, it was practical to build an 8-drive-in-2U chassis with one of these



That's a 3Ware Escalade 7800.

Now what you need to remember is that back in 2000, a 50GB SCSI drive like the ST150176LW was running around $830 wholesale, and while that Escalade card was around $500, Maxtor had released an 80GB for around $280, so a shelf of 9 50GB SCSI drives would run around $8000 for 450GB (including enclosure but no server) but a server of 8 80GB IDE's, complete with server AND controller, was only around $3800 - and also gave you 640GB of space. You could get two of those for less than the SCSI disk shelf alone.

The problem is that the cheaper IDE solution was a real bear to replace a drive, so I was really happy to see SATA arrive and we were an early adopter of the AIC 4U 24 drive chassis, which allowed us to pop in three 3Ware 9500S-12MI's with multilane cables and removable drive trays that could actually be hot swapped.

But if you want to really get an education on this, then you have to ask me about the impact of RAID controllers on the costs, etc., but that's a whole pile of knowledge that I don't have the time to regurgitate today. I spent more than a decade making this stuff work well, both in the hardware and in the software, mostly by finding new perspectives and alternative angles.

 

[cyberjock]

I have one of those 3Ware Escalade cards somewhere! LOL

 

[jgreco]

I have one of those 3Ware Escalade cards somewhere! LOL

Ok now you have to make a FreeNAS system outta one.

 

[cyberjock]

Ok now you have to make a FreeNAS system outta one.

Hell no! LOL.

 

[jgreco]

Hell no! LOL.

Didn't think you were up to it. I fondly remember the days of ordering a hundred PATA drives for a storage project... oh wait, ya, that was hell.

 

[Ericloewe]

Didn't think you were up to it. I fondly remember the days of ordering a hundred PATA drives for a storage project... oh wait, ya, that was hell.

I can only hope round PATA cables were used. The thought of traditional ribbon cables being used scares me.

 

[diedrichg]

hello all,

one of my coworkers has been telling me that I would be much better off going Usenet instead of my VPN Torrent approach to scraping shows and movies. I wanted to get some opinions and maybe even some war stories/experiences with anyone transitioning from using torrents to usenet.

I intend to use it in conjunction with Sickrage and Couchpotato that I already have configured using torrents through Transmission. I am toying with adding Headphones, but not sure how much music/genres/bands I would actually want to listen to is out there for me to grab. All of this appears to be pretty simple to toggle/swap between torrents and nzb so I am not really worried about it being an intensive shift. I suppose my main hesitation is finding a good indexer and service and then whether or not the potential additional cost per month over my VPN cost now is worth the gains/throughput.

Thanks in advance for the insight.

1. Find the Usenet provider that gives you the best bang for your buck. i.e. price, speed, d/l slots, bandwidth, SSL connection (a MUST for Usenet). For me, I went with Astraweb and bought the $50 1,000GB block. It's awesome! I have so far purchased two of these blocks and I'm well into my third year of using them. It's about 1.5 years per block, or less than $3/mo! My usage is 3-10 active TV shows per week and 1-3 movies per month. The initial library build ate a big chunk of data but it plateaued as the list of desired movies dwindled. I download TV shows in 720p and general movies in 720p with animated, sci-fi, c.g. movies in 1080p.

2. Purchase a backup provider who also has a small datablock but who is located in the Netherlands. They are slower to react to DMCA and therefore you will be more likely to complete a download (such as The Big Bang Theory) who frequently is missing "blocks" and can't be completed on Astraweb because of DMCA. You would set Astraweb as the primary server in SABnzbd and (SSL-News or Tweaknews) as the backup server. When blocks are missing, SAB will ask the backup server if it has the missing pieces to finish. With this setup I very rarely need to search for torrents.

3. I have a few invites to Newztown if you are interested. If I were to purchase an indexer I'd go with OZNZB.

 

[jgreco]

I can only hope round PATA cables were used. The thought of traditional ribbon cables being used scares me.

Round cables are impractical because you need varying lengths. We have all the supplies to make custom PATA cable assemblies that fit an application perfectly here...

 

[diedrichg]

Round cables are impractical because you need varying lengths. We have all the supplies to make custom PATA cable assemblies that fit an application perfectly here...

What's the max length? 1m, 10m?

 

[jgreco]

What's the max length? 1m, 10m?

Funny guy. 10M? This ain't LVD SCSI (12M). This is the crappy extension-of-the-AT-bus-to-the-HDD evolved from IDE.

 

[Bidule0hm]

Hum... I'm almost tempted to do a 10 m ribbon and test it on an old config just to see what happens

 

[jgreco]

I'd make you one but I'm guessing France makes it expensive to ship there ;-)