| 00:02:13 | | hooway quits [Client Quit] |
| 00:15:18 | | thuban joins |
| 01:53:05 | | ragu_ joins |
| 01:56:32 | | ragu__ quits [Ping timeout: 252 seconds] |
| 01:56:50 | | ragu__ joins |
| 01:58:24 | | ragu_ quits [Ping timeout: 240 seconds] |
| 02:00:01 | | Dallas quits [Client Quit] |
| 02:00:56 | | ragu_ joins |
| 02:03:24 | | ragu__ quits [Ping timeout: 240 seconds] |
| 02:06:08 | | Dallas (Dallas) joins |
| 03:02:42 | | VADemon quits [Ping timeout: 252 seconds] |
| 03:06:25 | | britmob joins |
| 03:09:18 | | britm0b quits [Ping timeout: 252 seconds] |
| 03:22:39 | | britm0b joins |
| 03:26:04 | | britmob quits [Ping timeout: 240 seconds] |
| 03:32:52 | | HackMii quits [Remote host closed the connection] |
| 03:33:13 | | HackMii (hacktheplanet) joins |
| 04:02:59 | | HackMii quits [Remote host closed the connection] |
| 04:03:17 | | HackMii (hacktheplanet) joins |
| 04:22:59 | | ragu_ is now authenticated as ragu |
| 04:28:24 | | qw3rty_ joins |
| 04:32:11 | | qw3rty quits [Ping timeout: 252 seconds] |
| 05:09:47 | <@arkiver> | mgrandi: what is the original format? |
| 05:09:52 | <@arkiver> | try to include the original format |
| 05:09:53 | <@arkiver> | and a PDF |
| 05:12:02 | <@JAA> | I'd guess Google uses some proprietary stuff for storage? In any case, not accessible AFAIK. |
| 05:19:42 | <@arkiver> | ah it's from google docs |
| 05:19:47 | <@arkiver> | right, I guess PDF would be fine then |
| 05:33:23 | | ragu__ joins |
| 05:37:05 | | ragu_ quits [Ping timeout: 252 seconds] |
| 05:43:29 | <tech234a> | Google has kix format... not sure if it's how they store it but I think it's used for rendering |
| 05:44:01 | <tech234a> | Also I don't think Kix files are particularly useful as I don't think they can be re-imported or rendered with third party software |
| 05:45:00 | <tech234a> | https://docs.google.com/document/export?format=[docx,kix,etc.]&id=[id] ex: https://docs.google.com/document/export?format=kix&id=1nPu6Wy4LWR66EFLeYInl3NzzhHzc-qnk4w4PX-0XMw8 |
| 05:54:04 | <mgrandi> | Never heard of kix, but yeah google has some weird internal format, simple files are probably fine but I know spreadsheets have varying degrees of export compatibility |
| 06:35:13 | | DopefishJustin quits [Remote host closed the connection] |
| 06:43:17 | | DopefishJustin joins |
| 06:43:18 | | DopefishJustin is now authenticated as DopefishJustin |
| 06:56:51 | | mutantmonkey quits [Remote host closed the connection] |
| 06:56:52 | | HackMii quits [Write error: Connection reset by peer] |
| 06:57:09 | | HackMii (hacktheplanet) joins |
| 06:58:13 | | mutantmonkey (mutantmonkey) joins |
| 07:29:34 | | DopefishJustin quits [Remote host closed the connection] |
| 07:32:44 | | HackMii quits [Remote host closed the connection] |
| 07:32:44 | | mutantmonkey quits [Remote host closed the connection] |
| 07:33:05 | | HackMii (hacktheplanet) joins |
| 07:34:04 | | mutantmonkey (mutantmonkey) joins |
| 07:34:55 | | DopefishJustin joins |
| 07:34:55 | | DopefishJustin is now authenticated as DopefishJustin |
| 08:42:58 | | ragu__ is now authenticated as ragu |
| 09:36:22 | | spirit joins |
| 10:40:40 | | hooway joins |
| 10:53:17 | | alright joins |
| 11:11:40 | | alright leaves |
| 14:33:53 | | Arcorann quits [Ping timeout: 252 seconds] |
| 15:13:35 | <OrIdow6> | So I would like to get started at some point on these .eu domains |
| 15:14:02 | <OrIdow6> | I suspect there will be a lot |
| 15:14:28 | <OrIdow6> | arkiver: I think you said you were going to check if IA was making any effort of their own on that, any response? |
| 15:18:42 | <OrIdow6> | Also will see about doing an estimate for Smackjeeves... hundreds of thousands of "titles" (comics), but very few are long |
| 15:19:07 | <OrIdow6> | Question is how few |
| 15:19:18 | <OrIdow6> | And also how heavy their horrible JS interface will be |
| 15:21:20 | <OrIdow6> | Or maybe there are less, ID doesn't seem to be sequential |
| 15:21:25 | <OrIdow6> | In any case... will find out |
| 16:08:47 | | britmob joins |
| 16:11:47 | | britm0b quits [Ping timeout: 252 seconds] |
| 16:16:28 | <@JAA> | Hah, funny, I just thought about the UK-owned .eu domains a few hours ago and was going to ask as well. :-) |
| 16:49:47 | | britm0b joins |
| 16:53:04 | | britmob quits [Ping timeout: 240 seconds] |
| 18:03:26 | | mutantmonkey quits [Remote host closed the connection] |
| 18:03:45 | | mutantmonkey (mutantmonkey) joins |
| 18:05:54 | | mutantmonkey quits [Remote host closed the connection] |
| 18:06:14 | | mutantmonkey (mutantmonkey) joins |
| 18:18:12 | | britm0b quits [Read error: Connection reset by peer] |
| 18:21:23 | | britmob joins |
| 18:22:36 | | britm0b joins |
| 18:26:42 | | britmob quits [Ping timeout: 252 seconds] |
| 19:41:28 | <OrIdow6> | So SmackJeeves title and author IDs are distributed weirdly - https://transfer.notkiska.pw/inline/6TW0J/SmackJeeves_User_IDs.png https://transfer.notkiska.pw/inline/kOE74/SmackJeeves_Comic_IDs.png (obviously I don't have many here, but on previous runs with a narrower range they've looked something like that as well) |
| 19:43:53 | <OrIdow6> | SO this may take some time - I'm trying random IDs in excess of what the range actually appears to be to be sure |
| 19:49:50 | <OrIdow6> | https://www.smackjeeves.com/help/4 says there are "over 300,000", but it looks to me like there's only about a tenth of that - wonder why |
| 19:50:28 | <OrIdow6> | Considering the great majority of things on platforms like this tend to be inactive, & that it isn't paid, user abandonment isn't too plausible |
| 21:30:06 | | lennier2 joins |
| 21:30:47 | | lennier1 quits [Ping timeout: 252 seconds] |
| 21:30:56 | | lennier2 is now known as lennier1 |
| 21:36:12 | | lennier2 joins |
| 21:37:33 | | spirit quits [Client Quit] |
| 21:38:28 | | lennier1 quits [Ping timeout: 252 seconds] |
| 21:38:38 | | lennier2 is now known as lennier1 |
| 21:44:24 | | lennier1 quits [Ping timeout: 240 seconds] |
| 22:04:34 | <tech234a> | Orldow6: Common Crawl released a domains list https://commoncrawl.org/2020/10/host-and-domain-level-web-graphs-julaugsep-2020/ |
| 22:05:08 | <tech234a> | they have 89 million domains, or 539 million if you include subdomains |
| 22:05:36 | <tech234a> | though I think they drop www |
| 22:05:46 | <tech234a> | but it should be a good start for finding eu domains |
| 22:09:14 | | Arcorann (Arcorann) joins |
| 22:09:55 | <OrIdow6> | tech234a: Discovering them isn't the problem, identifying & saving them is |
| 22:10:10 | | Arcorann quits [Remote host closed the connection] |
| 22:10:17 | <tech234a> | ah, need to find a connection to UK right? |
| 22:10:32 | | Arcorann (Arcorann) joins |
| 22:16:26 | <OrIdow6> | Yes |
| 22:18:47 | <tech234a> | Hmm... only ways I can think of is seeing if domains are publicly registered to someone with a British sounding name, a company incorporated in the UK, or hosted on a server with a British IP address |
| 22:18:58 | <tech234a> | But that probably wouldn't get everything |
| 22:20:47 | <OrIdow6> | WHOIS data is not public for .eu |
| 22:21:05 | <OrIdow6> | https://hackint.logs.kiska.pw/archiveteam-bs/20200926 for the previous discussion |
| 22:26:38 | <tech234a> | ah |
| 23:32:38 | <OrIdow6> | So I'm working on "LA Stampa" and apparently there are a million pages |
| 23:32:54 | <OrIdow6> | Think I've figured it out, writing the script |
| 23:33:04 | <OrIdow6> | But may need help running it when the time comes |
| 23:37:50 | <OrIdow6> | Maybe that's not pages, maybe it's articles |
| 23:39:09 | <jodizzle> | Oh great, I was wondering about that one. |
| 23:39:52 | <jodizzle> | I did it a quick look, and it seems like it was data blobs served out via a flash application? |
| 23:41:08 | <OrIdow6> | Yes, though the "blobs" are ultimately sent over HTTP |
| 23:41:36 | <OrIdow6> | Which obviously makes it a lot easier |
| 23:41:41 | <jodizzle> | Right |
| 23:41:50 | <jodizzle> | What format are they again? JSON? |
| 23:42:12 | <OrIdow6> | Most metadata & OCR looks to be in JSON, images are in JPEG |
| 23:42:34 | <jodizzle> | Right, and then the JSON references the image URLs, iirc |
| 23:43:35 | <OrIdow6> | Yes (though from what I've seen the URLs are predictable enough that you could generate them without the JSON, if that became necessary for whatever reason) |