HomeSearchPrevious dayNext day

00:07:51thenes quits [Ping timeout: 255 seconds]
00:09:36thenes (thenes) joins
01:29:03Guest quits [Quit: Guest]
01:29:20Guest joins
01:51:41cyanbox joins
01:56:38Guest quits [Client Quit]
02:04:34pabs quits [Ping timeout: 260 seconds]
02:07:05Guest joins
02:25:34saumon quits [Ping timeout: 260 seconds]
02:28:04saumon (saumon) joins
02:34:38pabs (pabs) joins
03:34:07DogsRNice quits [Read error: Connection reset by peer]
04:46:52cyanbox quits [Read error: Connection reset by peer]
04:48:38cyanbox joins
04:50:17cyanbox quits [Read error: Connection reset by peer]
04:52:13cyanbox joins
06:33:32Osiride (Osiride) joins
06:57:33lemuria quits [Ping timeout: 258 seconds]
07:04:14lemuria (lemuria) joins
07:09:50Osiride_ (Osiride) joins
07:12:30Osiride quits [Ping timeout: 258 seconds]
07:12:30Osiride_ is now known as Osiride
07:14:02lemuria quits [Ping timeout: 258 seconds]
07:25:44lemuria (lemuria) joins
07:56:19Osiride quits [Ping timeout: 260 seconds]
09:52:02Webuser284019 joins
09:53:44<Webuser284019>is there a way to clear uploaded items from the warrior image on the coimputer as i am running out of storage. only way i seam to be able to do such a thing is delete and reinport
11:00:01Bleo182600722719623455222 quits [Quit: The Lounge - https://thelounge.chat]
11:02:23cyanbox quits [Read error: Connection reset by peer]
11:02:49Bleo182600722719623455222 joins
11:04:07cyanbox joins
11:04:28Osiride (Osiride) joins
11:07:57cyanbox quits [Read error: Connection reset by peer]
11:08:51cyanbox joins
11:15:53cyanbox quits [Read error: Connection reset by peer]
11:16:48cyanbox joins
11:17:20cyanbox quits [Read error: Connection reset by peer]
11:19:16cyanbox joins
11:52:24<anonymoususer852>"is there a way to clear uploaded items from the warrior image..." I am pretty sure that is automated. Which version of warrior image are you running? I'm using warrior 4.1, and with one of the AT projects already stating that I've uploaded 3.8TiB, there's no way that would fit within the 60GB image that is the default disk size.
11:53:50<anonymoususer852>You could also perhaps try doing things like "docker rm ..." to remove container images, but I'm unsure of its usefulness.
11:59:09<anonymoususer852>I am running, or have ran other AT warrior projects, in addition to the one that indicates 3.8TiB on the leaderboard, two smaller ones in the range of a few hundred GiBs, a couple that may fit within default 60GB but I am doubtful it would be stored locally if that was the case. All these just further proves it's impossible to fit within the default disk size limitations if each and every
11:59:15<anonymoususer852>uploaded items are still stored locally.
12:04:43<anonymoususer852>There's also "docker stats -a" and perhaps the old, typically compressed logs in /var/log that you could also look into.
12:12:30Osiride quits [Remote host closed the connection]
12:15:52<BornOn420>"docker system df -v" should give a good view of where your disk space went
12:19:42<BornOn420>Most containers will be only a couple of MB. Only exceptions I've seen are youtube, mediafire, github
13:04:14Sluggs quits [Excess Flood]
13:08:28Sluggs (Sluggs) joins
14:10:49monoxane quits [Ping timeout: 260 seconds]
14:42:14<Webuser284019>thank you i am using version 4.1 and i am doing the youtube version. so i am not sure if the it does clear out after a set amount of time
15:16:27unknownsrc joins
15:18:27<BornOn420>It should. But targets are a tad slow ATM, so uploads are not that fast
15:19:01<BornOn420>(my youtube container was 3GB an hour ago, now it's 700MB)
15:51:55cyanbox quits [Read error: Connection reset by peer]
16:20:32Makusu quits [Quit: The Lounge - https://thelounge.chat]
16:20:45Makusu (Makusu) joins
16:25:09<anonymoususer852>I didn't think about that, Youtube project and the likes (e.g. Twitch) would likely be big, as you're dealing with multimedia files. In my cases, meta ads may run big but that's not as frequent.
16:28:08<anonymoususer852>Speaking of meta, on my end it's sitting at 2.15GB but I suspect that's also likely due to the amount of concurrency I have on that; 20 instead of 6.
17:50:14Guest58 joins
18:13:27<BornOn420>for sure. that 3GB youtube is a concurrency of 1, so basically one video (in highest quality + audio)
18:14:27<BornOn420>Right now I'm 'banned' and it's down to 4.25MB
18:22:52<@JAA>Webuser284019: Do you run the VM or the container? The VM disk image will grow over time, see https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior#The_Warrior_virtual_machine_is_using_up_disk_space,_even_though_it's_not_running_a_project!
18:23:48<@JAA>'Reimport' suggests to me that it's the VM, so yeah.
18:24:33<@JAA>The data *is* being deleted inside the VM, but that doesn't prevent the VM disk growth, and there's nothing the warrior can do about that really.
18:25:33<@JAA>The disk image will not grow beyond 60 GB IIRC.
18:40:22DogsRNice joins
19:03:20<Webuser284019>im using VirtualBox. i do wonder if its possible to install natively bare-metal on a machine
19:06:46<@JAA>That's not supported, but you can use the container with Docker or Podman.
19:09:08<Webuser284019>ah, thank you. Looking through the info. it looks like the machine has a max size of 60gb so i guess it may delete uploaded stuff after a while.
19:12:38<@JAA>No, like I said, it's deleting completed items immediately. That just doesn't necessarily reflect in the disk image size, which is an artefact of how virtualisation works.
19:20:43<Webuser284019>ah ok so its a fixed size of 60gb but will not balloon up and up like crazy?
19:20:49BornOn420 quits [Remote host closed the connection]
19:21:08Webuser284019 quits [Quit: Ooops, wrong browser tab.]
19:21:25BornOn420 (BornOn420) joins
19:22:30<@JAA>Oops, wrong browser tab.
19:30:26Webuser543463 joins
20:33:12BornOn420 quits [Ping timeout: 255 seconds]
20:50:04BornOn420 (BornOn420) joins
20:58:21BornOn420 quits [Read error: Connection reset by peer]
20:58:55BornOn420 (BornOn420) joins
21:05:08Wake8 joins
21:08:56Wake quits [Ping timeout: 258 seconds]
21:08:56Wake8 is now known as Wake
21:14:49BornOn420 quits [Read error: Connection reset by peer]
21:15:22BornOn420 (BornOn420) joins
21:57:03<anonymoususer852>"... a fixed size of 60gb but will not balloon up and up like crazy?" That's answered here, https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior#How_much_disk_space_will_the_Warrior_use? if I am not mistaken.
21:57:49<anonymoususer852>"because each project defines items differently, sizes may vary. A single task may be a small file or a whole subsection of a website. The virtual machine is configured by default to use an absolute maximum of 60GB, but Docker has no hard limit. Any unused virtual machine or Docker container disk space is not used on the host computer."
22:02:25<anonymoususer852>I believe that should also cover the fact that YMMV if you intend to increase the concurrency. BornOn420 stated 3GB youtube is a concurrency of 1, therefore if you intend to run the same youtube project but upwards to 20 concurrency, you could be potentially witnessing 60GB being chewed up, due to 20 x currency with each one taking up 3GB, theoretically speaking.
22:04:03<anonymoususer852>s/currency/con&/
22:09:46<@JAA>Pretty sure the problem here is fragmentation of the virtual disk file.
22:10:06<@JAA>Which is kind of explained in the question I linked, albeit not in detail.
22:22:26<anonymoususer852>Ultimately it shouldn't exceed the hard limit. It will grow one way or another, but unlikely beyond that hard limit, or that would be a potential huge flaw with the VM software itself :).
22:34:54opl quits [Read error: Connection reset by peer]
22:35:09opl4 (opl) joins
22:52:02BornOn420 quits [Remote host closed the connection]
22:52:40BornOn420 (BornOn420) joins
22:54:04osiride|m joins
22:54:44BornOn420 quits [Max SendQ exceeded]
22:55:16BornOn420 (BornOn420) joins
23:01:01BornOn420 quits [Remote host closed the connection]
23:02:27BornOn420 (BornOn420) joins
23:04:29BornOn420 quits [Max SendQ exceeded]
23:05:04BornOn420 (BornOn420) joins
23:07:09BornOn420 quits [Max SendQ exceeded]
23:07:44BornOn420 (BornOn420) joins
23:09:50BornOn420 quits [Max SendQ exceeded]
23:10:23BornOn420 (BornOn420) joins
23:55:49BornOn420 quits [Remote host closed the connection]
23:56:27BornOn420 (BornOn420) joins

HomeSearchPrevious dayNext day