Being an Amateur Radio Operator I use and beta test a small device called a TinyTrak4. Now there is a support group for the development of this small device, hosted on Yahoo Groups. As most of us would be aware, Yahoo Groups has many adverts embedded into their mailing lists. Now I thought I’d be cleaver and set-up Mailman on my personal server and mirror the Yahoo Group list into my Mailman mailing list. Well the idea was a good one in thought, but just recently I have just discovered an error in my way.
To start off with I was getting a copy of the target mailing list with a small perl script called yahoo2mbox.pl so as the title implies it simply grabs the yahoo mail list of your choice and saving the output as a mbox formatted file without getting all those unwanted adverts! I went about this the following way…
./yahoo2mbox.pl tinytrak4 –user=user –pass=password && cp tinytrak4 tinytrak4.mbox && chmod 664 tinytrak4.mbox && chown www-data:list tinytrak4.mbox && cp tinytrak4.mbox /var/lib/mailman/archives/private/tinytrak4.mbox/tinytrak4.mbox && mmarch tinytrak4 && echo Import Complete!
Can you spot the error? OK lets break the above down into steps, firstly we invoke the script then advise of the mailing-list we want to grab the content from, then passing a valid user and password. This then created an mbox formatted file (in this case tinytrak4) this was then renamed to include the .mbox extension and the permissions & ownership altered to match the default used by Mailman and then copied to the location used for this mailing-list… an now for the big mistake… by executing mmarch, this then causes a complete rebuild of all archive pages without removing the previous archive run thus accumulating multiple copies of the same information! So over time this is just eating up hard disk space.
So a lesson here is here that I should have seen this problem after a few runs of this script because of the excessive hard disk space being consumed… sadly I didn’t pick up on this until now (nearly a year after running this once a day!) 😐
I have since removed this mailing list from Mailman regaining 77% storage space back! (34GiB total storage for this server with 1.5Gib set as swap space)