00:24:00<dashcloud>SketchCow: did you find yourself needing to limit how fast you downloaded from FTP sites, or is that not really an issue?
00:53:00<SketchCow>No.
00:53:00<SketchCow>Most are so dead they don't notice.
00:56:00<SketchCow>One trick I'm finding that is working is searching for certain magic words, specifically dead famous FTP sites, means I'm finding mirrors and in those mirrors are often mirrors of OTHER websites.
00:56:00<SketchCow>Example: https://www.google.com/search?q=ftp+garbo
00:56:00<chronomex>it's mirrors all the way down
00:57:00<SketchCow>http://darkwing.uoregon.edu/~joe/best-ftp-sites.html see?
00:58:00<SketchCow>http://153.19.251.222/mirror/ has shitty mirrors
00:58:00<SketchCow>But that's a start
00:58:00<SketchCow>and so on and so on
01:03:00<dashcloud>here's a copy of chapter 10 from Internet Games Directory- a listing of what FTP sites were good then: http://pastebin.com/NA610GXe
01:04:00<dashcloud>(the book is from 1996)
01:06:00<dashcloud>I've started grabbing Halycon and the first mirror listed for it
06:33:00<soultcer>chronomex: Could you take a look at the tracker host? It's been down for over a day and we need it to finish the webshots stuff. Maybe it's just a simple problem that can easily be fixed
06:33:00<chronomex>sure
06:33:00<chronomex>22:33:39 up 1 day, 12:39, 1 user, load average: 1.05, 0.96, 0.96
06:33:00<chronomex>you sure?
06:34:00<Deewiant>http://tracker.archiveteam.org/webshots/
06:35:00<chronomex>hmm, alard set that up, and I don't know how he configured it
06:36:00<soultcer>alard hasn't been around the last two days, maybe he is busy with work/private life
06:36:00<chronomex>k
06:37:00<trythil>I've set up tracker before
06:37:00<trythil>and can take a peek
06:37:00<trythil>oh, yeah
06:37:00<trythil>I know what's wrong there
06:37:00<trythil>Redis is just offline
06:38:00<chronomex>ok cool, know what I should do from here?
06:38:00<chronomex>oh spiffy, there it goes
06:38:00<yipdw>if there's a script to start Redis, it just needs to be restarted
06:38:00<chronomex>thanks!
06:38:00<yipdw>np
06:39:00<yipdw>I wonder why it crashed
06:39:00<yipdw>was it OOM-killed?
06:39:00<chronomex>good question
06:39:00<soultcer>What kind of VPS is the tracker running, btw?
06:40:00<chronomex>it's on a linode
06:40:00<chronomex>linode 512 to be precise
06:40:00<soultcer>Yay, redis came up clean this time for all trackers it seems
06:41:00<yipdw>Redis can be pretty memory-hungry
06:41:00<soultcer>Damn that Linode thing is expensive
06:41:00<yipdw>and 512 MB is probably pushing it
06:41:00<yipdw>so
06:41:00<chronomex>if you have suggestions to move it elsewhere, soultcer, I'm game
06:41:00<chronomex>but my laziness outweighs a small amount of cost savings
06:42:00<yipdw>the tracker does generate a lot of data, but last I checked it doesn't generate that many keys
06:42:00<yipdw>and keys are what you can't swap to disk in Redis
06:42:00<chronomex>plus, linode gave me a sweet tshirt at sxsw
06:42:00<yipdw>maybe that changed; I haven't looked at the tracker code since alard added multi-project capability
06:42:00<chronomex>no, there are still only few dozen/hundred keys
06:42:00<yipdw>oh
06:42:00<soultcer>I thought redis remove the swap-to-disk feature again
06:42:00<yipdw>yeah, that should be fine
06:43:00<yipdw>oh, so it was removed
06:44:00<yipdw>well never mind then, I guess we do have to take into account value size
07:08:00<ivan`>http://thenextweb.com/google/2012/11/25/google-shuts-down-dedicated-motorola-mobility-sites-for-asia-europe-middle-east-and-africa/
07:40:00<Nemo_bis>Too bad, it rains.
07:40:00<Nemo_bis>I planned to reach the far special post office which allows me to send SketchCow those magazines by bike.
07:49:00<godane>i uploaded 6 issues of pctoday magazine so far
07:49:00<godane>http://archive.org/details/pctoday-magazine-v2i6
07:50:00<godane>its only issue 6 thur 12 of volume 2 so far
10:24:00<chronomex>yipdw: looks like redis shat the bed ~an hour ago, i just kicked it back into life
10:24:00<chronomex>i am going to bed and do not have high hopes
10:25:00<SmileyG>lol that might explain my issues.
10:25:00<SmileyG>giff access?
10:26:00SmileyG realises this is his lifes objective
10:26:00<SmileyG>to run stuff like this for other people
10:28:00ersi rolls eyes
10:30:00<SmileyG>:O
10:30:00<SmileyG>hmmm restarted warrior now works :/ weird.
15:16:00<closure>was someone archiving ftp.funet.fi recently? I'd like to get a copy of the usenet archive portion
15:20:00<Deewiant>closure: <@balrog> oh yeah, the archives I'm uploading include a copy of ftp.funet.fi from 2008
15:22:00<closure>did he say where?
15:23:00<Deewiant>Nope. And that was almost a week ago.
15:24:00<DFJustin>iirc they're going onto an IA server where sketchcow will then package them up into items
15:36:00<balrog_>closure, Deewiant: uploaded it to SketchCow's box
15:37:00<SketchCow>1989_01_BYTE_14-01_PC_Communications_and_Annual_Awards_and_Digitizing_Tablets.pdf
15:37:00<SketchCow>WHEEEEE 1989 of 03 and Volume 14 and 03
15:37:00<SketchCow>1989_03_BYTE_14-03_Mac_Supplement_286_vs_386sx_Object_Oriented_Programming.pdf
15:37:00<SketchCow>WHEEEEE 1989 of 04 and Volume 14 and 04
15:37:00<SketchCow>1989_04_BYTE_14-04_CASE_and_UPSes_and_Graphics_Suppliment.pdf
15:37:00<SketchCow>etc
15:37:00<SketchCow>Allow me a little to upload 4.3 gb of Bytes
15:37:00<SketchCow>Then we'll get to the other
15:45:00<ersi>chronomex: Something seems to be up with the tracker again, etc. (Meant to write that in this channel)
15:52:00<SmileyG>its on fire.
16:07:00<SketchCow>There it go
16:07:00<SketchCow>(The Byte Upload)
16:18:00<ersi>SmileyG: Thanks for your comment.
16:18:00<SmileyG>ersi: quite alright.
16:18:00<SmileyG>Its licenced cc-by-sa
16:19:00<SketchCow>Aaaaand up they go
16:24:00<SketchCow>The deriver is NOT going to like those. 200mb in some cases.
16:25:00<chronomex>ersi: ugh, gimme a few hours to wake up and go to work and shit
16:27:00<ersi>chronomex: Sure thang. Just thought I'd poke you the earlier the better :)
16:27:00<chronomex>yeah, thx for adding it to my agenda
16:39:00<chronomex>just kicked the server, will investigate later
16:39:00<ersi>coolers
16:39:00<chronomex># /etc/init.d/redis-server restart ; /etc/init.d/nginx restart
16:39:00<chronomex>if that doesn't fix it, I don't know what else will!
16:43:00<chronomex>Out of memory: Kill process 21534 (redis-server) score 284 or sacrifice child
16:43:00<chronomex>Killed process 21534 (redis-server) total-vm:309624kB, anon-rss:111352kB, file-rss:260kB
16:43:00<chronomex>oh yep
16:43:00<chronomex>fuck
16:44:00<Aranje>sacrifice child: the only logical option
16:45:00<SmileyG>nom nom nom, that mem tastes good.
16:45:00<SmileyG>How much mem is it eating btw?
16:46:00<soultcer>Does linode allow you to upgrade RAM?
16:46:00<chronomex>yeah, but you have to upgrade the whole thing at once
16:47:00<chronomex>300M, according to that whine line
16:47:00<SmileyG>300Mb... how much in the box~? :S
16:47:00<SmileyG>generate giant /swap
16:48:00<chronomex>the box is only 512M
16:49:00<SmileyG>:/
19:12:00<SketchCow>The Byte magazine is STILL being derived.
19:12:00<SketchCow>Go team
20:05:00<SketchCow>http://techcrunch.com/2012/11/26/twitter-ceo-dick-costolo-twitter-sees-a-billion-tweets-every-two-and-a-half-days-users-can-download-their-entire-archive-by-year-end/
20:13:00<SmileyG>o_O
20:15:00<swebb>I seriously dout it though.
20:15:00<swebb>doubt it
20:33:00<ersi>Haha, Dick. Great name.
20:33:00<SmileyG>:D
20:54:00<SketchCow>http://archive.org/details/ftpsites
21:34:00<bsmith094>for file renames, i apparently need to use regexps, (at least for anything other than basic ones) and i cant figure this out. i want from the beginning of the name to the first instance of " - " removed, minus the quotes
21:35:00<godane>SketchCow: i upload my iso yesterday and its still not in http folder: http://archive.org/details/cdrom-linuxformatmagazine-142
21:35:00<godane>please don't tell me it go lost in the mail
21:35:00<godane>takes 5 hours just to upload it
21:36:00<godane>ok i know why
21:36:00<DFJustin>if you see the word "torrent" crossed out on the left that means it's still processing
21:36:00<godane>its waiting to run
21:37:00<godane>its just weird that i got 2 items waiting to run
21:38:00<DFJustin>processing times are highly variable depending on the phase of the moon and how much clop fanart underscor is running through the servers
21:39:00<godane>good news is none of my pctoday magazines are not breaking
21:39:00<godane>or stop derive cause its having a problem
21:40:00<underscor>:D
21:40:00<godane>DFJustin: Is the iso losted?
21:41:00<godane>you ran it and there is no iso in http folder
21:41:00<godane>https://ia601508.us.archive.org/34/items/cdrom-linuxformatmagazine-142/
21:41:00<DFJustin>hit refresh
21:42:00<DFJustin>also I ran nothing
21:42:00<godane>it must have been underscor or someone here
22:10:00<SketchCow>http://archive.org/details/byte-magazine-1989-04 took 6 hours to process
22:55:00<godane>is archive.org having connection issues?
23:02:00<godane>pctoday magazine is uploaded
23:02:00<godane>up to end of 2011
23:35:00<Cowering>SketchCow, are you missing any BYTE mags? I probably have at least 10 years of them in the attic