Archiving the ezBoard RTc forums

Post by **Gambit37** » Wed Apr 30, 2003 4:14 pm

George, with your permission I'd like to archive the posts on the RTC forum. I know that you're intending to keep the forum there for references, but I think it will be useful to keep an offline version handy somewhere just in case...

I've been testing the ezBoard/phpBB converter that someone mentioned in another post. It works very well indeed. I'd like to use this script, but for it to work correctly on an ezBoard forum, these steps have to be undertaken:

1. Re-open all closed topics in ezboard; the script does not convert closed topics.
2. Un-sticky all sticky topics; sticky topics cause the script to convert only the first page of forum topics.
3. Set the display style of your ezboard forum to ezBoard Classic.

If you're agreeable, could you therefore make these modifications and I will archive the data over the next few days. Once the data has been archived, I can either maintain it myself, or we can easily move it over to Cosmanauts forums as it will be in the correct format.

Post by **ChristopheF** » Wed Apr 30, 2003 7:04 pm

great!

I suggest that you import the old posts here directly in the correct forums.
Then we can really get rid of the ezboard forums (just leave a link pointing here). The archive will be easily accessible, and people will be able to directly continue threads started on the other board.

Also importing Ian's forums here would be very nice.

George Gilbert · Post by **George Gilbert** » Thu May 01, 2003 10:27 am

Gambit - all done as suggested; although there were no sticky topics and it was already in "classic" style which made it somewhat easy ;-)

Let me know when you're done and I'll shut it all back up again to ensure people post here rather than there.

thanks for sorting this out!

Post by **Gambit37** » Thu May 01, 2003 1:11 pm

OK, I now have all the data archived from the ezBoard forums, so yoiu can lock it off again George.

The forums are currently available for viewing here:

http://www.dmcodex.com/forums/index.php

They are in a raw state and contain some junk (smilies came out weird), but it's pretty much an accurate copy of what was on ezBoard. The question now, is what to do with it all?

If you want all this uploaded on to your forums Cowsmanaut, I think I'd need to make some changes to it first. The data is about 2.6MB, with 138 users. I reckon it would be best to do this:

1) Change all the user names so that they begin with RTC_, this way there won't be any conflicts when uploading the old users posts.
2) Prune out all unnecessary posts, things like off topic notes, warnings, or pointless threads that weren't replied to.

However, I'm not sure if it would be best to try and add all the old posts to the current RTC forums on this site, or to create an entirely new category called RTC ARCHIVE and put all the data in there. Either way, I will need all your login details, and mySQL passwords etc. It's quite a big job, and I'm happy to do it if you trust me to do so.

Everyone's thoughts on the matter will be appreciated.

Post by **ChristopheF** » Thu May 01, 2003 7:03 pm

As I said, and if it not too much trouble, I'd prefer to see the old posts imported in the current RTC forums here, not in a separate catergory of "old RTC forums". The RTC forums here are a continuation of the old ones, so if it is possible to merge them, I think it is better.
It might be difficult to import the posts in a non empty forum here... I don't know...
If it is, then you can perhaps import the old posts in new forums, then move the posts from the current forums here to these new forums, make some rename / delete so that at the end we only have one set of RTC forums with everything inside. What do you think?

Post by **Gambit37** » Thu May 01, 2003 7:19 pm

That seems like the most practical solution. I would like to consolidate old users posts with their new login details, and that's the thing that would take the most time. It would require very careful manipulation of the database.

Thinking on my feet, the easiest way to do it is for me to take a local copy of Cowsmanaut's current phpBB database, merge all the old RTC data with the database, and re-upload the database. For this to work, these forums would have to be disabled for however long it takes mae to do the the work; I estimate about 3-4 hours. I will also need access rights to the mySQL database so that I can overwrite it. Cowsmanaut, how do you maintain the mySQL database for this forum? Are you using something like PHPAdmin?

The more information I can find out about how this forum is setup, the better I can decide on how to undertake the task...!

Post by **cowsmanaut** » Thu May 01, 2003 8:12 pm

Agian, I'm waiting for access to be returned to me for the FTP where you can access the files. As for any of the SQL, I do anything I've done either through admin on the net or through FTP access.

beowuuf · Post by **beowuuf** » Thu May 01, 2003 9:13 pm

Out of interest, can you move the order of posts in a forum?
If you ad old posts to the forums, can you be sure they won't flood the new posts that are already here?

Otherwise i'd say go for it, and yes, try to integrate the forums as much as possible so the suggetsions has suggestions, etc...

Post by **Gambit37** » Thu May 01, 2003 10:02 pm

OK, I realise that I can't do any of this until you have found out the access passwords. But I'm speaking speculatively at the moment.

It's not really going to be possible to do any of this unless I can get direct access to the administration tool that you (Cowsmanaut) use to administer your database. When you installed phpBB, the installation procedure asked you to tell it which database to use -- this is a database that you would have previously set up using either mySQL directly or some kind of admin tool (usually provided by your ISP as part of your hosting package). I'll eventually need to know all your lgin and admin details for this tool to be able to integrate the old posts. Make sense?

Post by **cowsmanaut** » Fri May 02, 2003 6:21 am

you would be assuming I set it up in the first place. It was installed and there already for all users of the subdomains. All that was required was to branch off for mine.

so all I needed to do is set up the forums and accounts as well as install the style sheets etc.

beyond that I didn't need to do much.

moo

Post by **ChristopheF** » Fri May 02, 2003 9:44 am

Gambit, if there is no tool to administer the mySQL database, then you may be able to use the backup/restore feature of phpbb:
1) backup the database
2) restore it "offline" on your own computer
3) merge with the other database you have from the RTC forums
4) restore the new database here on this site

Post by **Gambit37** » Fri May 02, 2003 11:43 am

@beowuff: the old posts retain their original posting dates, so once they are merged, it'll just look as if people have been posting here since 2000...

@cowsmanaut: ah, no problem then! I will have to find an alternative way around this, which might mean....

@christophe: .....doing exactly as you suggested! That's obviously the best solution, and I've just done a quick test to see if it'll work: yes, it works fine. I can do all the work merging the databases locally, then when I'm ready to upload, it'll be reasonably easy...

The way I'm going to manage user names is as follows: For users who have posted under the same name in both forums, I'll keep the names the same. Where users have posted under different names (such as you Cowsmanaut, you used Drake, 0_drake_0, etc.) I will try and consolidate all the posts to the one user name, but this might take a very long time due to the nature of how the data is stored. For all other user names, I will add a prefix (any suggestions? perhaps just RTC_) to the user name so it's clear it's from the old forum.

The big problem is the posts themselves. They all use unique ID's and lookup tables to determine who's posted each message. Now, it's going to be a big job organising this as obviously both databases will have posts with the same ID, so I'll have to increment all the RTC posts to use IDs that are much higher than anything in the other database. Then manage the lookup tables accordingly.... this in itself could take 2 or 3 days.

I'm unemployed at the moment, so have some "free" time. I'll start looking at this today to see how complex it all is. Once I have a better idea of how long it'll take, I will report back.

At that point, THESE FORUMS MUST BE CLOSED UNTIL THE NEW DATA IS COMPLETELY UPLOADED. I don't want to have to contend with the database changing on-line while I'm working on an older version offline!This could mean they are closed for a few days. If you can all cope with that, then that's what I'll do....

BTW, I spent a few hours yesterday working on the ezBoard data and have stripped out about 300Kb of ezBoard markup and have also replaced the ezBoard smilies code with phpBB smilies code, so it all looks a lot better now!

Post by **Gambit37** » Fri May 02, 2003 1:42 pm

Well, I've had a look at the structure of the data, and it looks reasonably straightforward to merge the databases together. However, I'll have to do it manually, which is going to be very time consuming, probably a couple of days altogether.

I'd like a group agreement on when we can close the Forums so that I can go ahead and do this. If it helps, I can work on it RIGHT NOW for a few hours, and also over the bank holiday here and there, so I estimate that if we closed the forums at sometime today, I could have it all ready for next Tuesday. The problem is that Mr.Cow who has the final say is probably asleep right now (time differences!)...

Oh, one quick note: one result of adding the old posts is that some people's post ranks will increase rather quickly....!

Post by **Lunever** » Fri May 02, 2003 3:50 pm

Well, as long as there is some remark when the site will be probably reopened again I don't think it matters when it will be closed. The merging of the old and the new forum is a really good thing, and I'm ready to wait a few days if nessecary to get this done. Thank you for investing time and work into it, Gambit!
As for the ID's: Well, as long as there are no mandatory technical reasons to do so I don't see why to give them any prefix at all. For those people who used the same ID in the old forum like in the new one it is very handy to have their old posts imported to their current ID. I mean, if you are looking for some specific thread by clicking on functions like "show all threads of this member" it doesn't make sense if you get the new ones only, have to notice that what you're looking for isn't there and then have to check another ID. Of course it wouldn't be a big problem, but it would be still unnessecary effort. As for those who changed ID's it would rather depend on whether you are willing to take the effort of manually merging them, but if you are willing to do so, for them same goes as above.

Another thing: I don't think any posts should be filtered out as "irrelevant", for that's how a forum is supposed to work.

Post by **Gambit37** » Fri May 02, 2003 4:44 pm

It's not that simple; a user ID in the phpBB database is just a number, and the name is irelevant. For example, "George Gilbert" is George's user name that we see on the forums. But internally he has an ID of 3. Now, in the new forum that I've created from the RTC posts, George's internal ID is 5. So before I can merge the data, I need to ensure that all the messages in the old forum with and ID of 5 are changed to have an ID of 3.... see what I mean? This is what will take the time as there are thousands of messages. And because I'm only a newbie when it comes to mySQL, I'll probably do it by hand, which will take even longer...

I also need to cross reference all message IDs from both forums, and modify all the numbers in one of the databases by incrementing them by a set number, so that there are no duplicates. It is by no means a quick job, and there is no easier way of doing other than forgetting the whole thing and simply giving all the old names a prefix.

Regarding irrelevant posts, I have only deleted things like George's "Forum moved" notes, and messages from people that said things like "I can't access such and such website, oh wait, yes I can"...

I will also need to delete some of the duplicated threads too, such as "Fighter gem", "CSB Dungeon mechanix" etc. because these already exist in the current forum.

Post by **Lunever** » Fri May 02, 2003 5:02 pm

Ok, then, if there is no solution to automatize this process, I can see why you would prefer a prefix. Just do what is possible within the time you are willing to invest into that problem. To have the old posts imported here with a different ID is surely better than to have them not imported at all.

Post by **ChristopheF** » Fri May 02, 2003 8:40 pm

Thank you for your time investment, Gambit.
I think you can shut down the forums just when you want, we can live without them for a few days... (don't forget to make a backup of the current database before changing things, in case it goes bad

)
And there are still the encyclopaedia forums during that time (performance is back to normal. well at least for me...).

Post by **Lunever** » Mon May 05, 2003 9:56 am

Hey, good work, Gambit! Great! Thank you!

One minor thing: Due to the many EZ_ID's (including those of many users who have'nt been regulars anymore even at ezboard) the memberlist looks somewhat confusing. Since it is sorted by subscription date, with oldest on top (if you don't change the sorting criteria) , you'll see all those EZ-ID's first, mixed with the current ID's (even if you do change the criteria). Those users whose EZ_ID couldn't be linked to an existing ID of the new forum retained their original subscription date, while those ID's which were merged got the new subscription date of their current ID's from the new forum. So:

1) Can we make the memberlist to show the current ID's on top of the EZ_ID's ?

2) Can we give the merged ID's their original subsription date from ezboard (since this would also help sorting the memberlist)?

PicturesInTheDark · Post by **PicturesInTheDark** » Mon May 05, 2003 12:06 pm

Looks like you did a great job, Gambit, thank you very much. We'll have a much better overview with only one combined forum and hopefully more people coming here.

Is there still a problem with the profiles, since I cannot get my usual "subSilver" style or am I doing something wrong?

Regards, PitD

Post by **Gambit37** » Mon May 05, 2003 1:46 pm

To everyone: You're welcome!

@Lunever: I haven't done anything with the member IDs from ezBoard so they only have data associated with them that the ez/php converter looked at. If that data is missing or innacurate, so be it. I'm not going to go back and fiddle with that data, it's just too time consuming. As for the sort, I guess I could look at the code and change the default sort order; I'll have a look.

@Picturesinthedark: At the moment, I've set the board to override users style selection. This is because the changes I've made have only been done to the Chronicles template. Because of the way the forums work, there are a number of PHP and TPL (HTML) files that require modification. I'll have a look at adding these changes to the other templates, and will let you know of any progress

Post by **Lunever** » Mon May 05, 2003 2:02 pm

@Gambit: NP, you did a great job!
I was just asking, because due to this dating it looks in the mamberlist, as if the oldest and still active members were also the youngest, but it is certainly not important enough to invest much time of manual work into it, so just forget it.

Post by **Gambit37** » Mon May 05, 2003 2:22 pm

@PITD: I've updated the subSilver template with the same changes I've made to Chronicles, and reset the board so that users can choose their own style.

@Cowsmanaut: I've noticed that you still have the original darker template "Dungeon Master" available (the one with the blue text). However, it appears to be severely broken

and would take quite a lot of time to fix. Is it OK if I remove it from the list of available templates?

@Lunever: There's something weird going on with the memberlist, because if you sort it either ascending or descending by Joined date, the list doesn't change! I'm therefore not going to bother investigating this because if the original phpBB code is flawed, I don't want to risk messing it up further!

Post by **Lunever** » Mon May 05, 2003 2:36 pm

That's strange indeed. But nevermind, I found using "number of posts" descending quite decent to filter most inactive members. Maybe that should be the default setting (only, if that can be changed without mesing with the code, of course).

Post by **cowsmanaut** » Mon May 05, 2003 5:51 pm

Gambit, yes you can ditch it.

PicturesInTheDark · Post by **PicturesInTheDark** » Wed May 07, 2003 3:37 pm

Thank you for the update, hopefully this will postpone the time when I will finally need glasses

Regards, PitD