GameFAQs Board Databases

From: Ngamer | Posted: 15/03/2006 1:31:28 AM
This afternoon I approached YE with a proposition for an extension to the standard userdb. He's thinking about how to implement it right now, but I wanted to make the idea public first, to get some outside viewpoints/suggestions.

The Idea: User databases limited to particular, active GameFAQs message boards.

The Example: Let's say that I make a post in the official topic ("New Board Database Suggestions") and say that I would like to begin work on a userdb limited specifically to Board 8. YE generates a new folder at with a layout identical to that of the standard userdb page. The only difference is that...

- we start from scratch, an entirely fresh table. this means there are 0 B8 users to start, and also that everyone is equal in terms of fetch counts.
- the fetch script is modified so that no topic can be searched unless it contains the string genmessage.php?<b>board=8</b>&topic=xxx.

By making use of this implementation we could easily construct complete records of all currently-active members of LUE, CE, RI, and the like. Think how many times you have seen "Post your user ID here!" topics on GameFAQs boards, with everyone pasting themselves into a woefully incomplete listing. We have the technology to create a complete, entirely accurate user listing at the snap of a finger; it's something we should consider taking advantage of.

The Catch: Every time a new LUEser, B8er, CEer, etc is discovered and added to the board-specific userdb, that username and ID # is also compared against the latest records in the full userdb. If we have a hit (not found in either db), the user will be added to both LUE and the complete database. Therefore, not only are you putting up huge fetch totals (think about it- it will be a return to the good old days, when you could find 400+ new users in a single topic!) as you create the LUEserdb, but you are also helping both the full userdb and your main fetch count by proxy. Talk about killing two birds with one stone!

The Summary:

Pros: - increased userdb activity
- increased userdb promotion ("hey, check it out, a complete list of active Board 8ers! are you on the list?")
- increased additions to the overall db
- a chance to start fresh on fetch counts
- a return to the days when fetching was extremely easy and very rewarding

Cons: - none
(bandwidth/storage space has not and will not be an issue, largely because we track so little info about each user)

What are your thoughts, gents?
From: Ngamer | Posted: 15/03/2006 1:36:01 AM
True, for Board 8 and others, we would lose the ability to work toward "100% of all users fetched". That functionallity could still stay in place for LUE, however, because as you'll recall we have a reliable ceiling to reach for there.

On April 29, 2005, CJayC said, in The Great GameFAQs Q&A, that "22,584 accounts originally signed up for LUE. Of those, 17,193 remain, the rest being lost to bans and purges."
From: Ngamer | Posted: 15/03/2006 1:45:05 AM
On the topic of board fetch counts, you could argue that, since we're working with smaller communities, it "wouldn't be fair" that whoever searched first would easily be able to fetch the lion's share of active users, gobbling up all the 500 and stickied topics and leaving everyone else the scraps.

So perhaps we could even the odds by having the creation of new Board dbs announced in advance. Example: YE puts up a topic stating "The LUEser database will go online Saturday, March 18th at 7:00PM EST." Suddenly, adding users to the database becomes an event- everyone wants to be there to snatch up the big topics and build huge fetch counts in the early going. Fetching will be so easy, even non-regulars might want to get in on the action.

And remember that all this time, there will be a strong potential for many new additions to the complete userdb.
From: Ngamer | Posted: 15/03/2006 2:05:01 AM

In order to grow the userdb, it is important that we concentrate on smaller, non-social boards with less-defined userbases. Boards like those belonging to the current Top Ten FAQs, where new users sign up to ask a question relating to the game. Placing the focus on popular social boards will limit the userdb's growth!

This is a good point. However, I think that this objection does not take into account what is likely an enormous amount of lurkers (LUEkers, if you insist) on these very-active boards. No one fetches many topics on a board like CE anymore because it's discouraging to run so many topics through and not find a new user. But with the increased activity, entire boards are going to be canvased time and again, with no one being discouraged because they're all making solid progress on completing the Board db. This action is bound to turn up many lurkers/infrequent posters who would otherwise have slipped through the cracks.

Or to look at it another way, you're right, game boards are the best place to make userdb additions. That being the case, our most 'hardcore' fetchers can continue that focus, adding to their already-ridiculous overall fetch totals. The board fetchings are so easy that anyone can do it, so they can be left to the more 'casual' fetchers. Thus we can have the best of both worlds- the Board dbs cover the lurker aspect nicely, while guys like Rock and Luster and Goomba and Monkey ignore such "kid's stuff" and continue rooting out the hard to find new users.
From: RockMFR | Posted: 15/03/2006 3:41:55 AM
Great idea. Just one thing I'm thinking about- the fetch script on the main page should contribute to the individual board databases. For example, if I'm fetching a topic on LUE on the main page, it should add the users to the LUEdb as well as the main db (I think you said the opposite of this- just want to make sure that it would work both ways).

The concept itself is very good. I can see this becoming very popular on a lot of boards. Though, as popularity increases, there will be issues that go along with this. Just a few of the *possible* things that I'm thinking of:

-bandwidth issues on this site
-complaints from GF administration (undoubtedly, there are going to be a few people who attempt to use a robot to fetch topics for the database). I'm not sure how closely they monitor page loads, but if this project starts fetching 20k or 30k topics every day, it might get on their radar, even if it's just from the popularity and nobody uses a bot. Though, the more I think about it, the more I don't think it will be that big of a problem. The GFBrowser on the vtaku site used to load a ton of pages (plus getting raped by the Google bot when it was public), and I don't think their were any complaints about it.
-privacy complaints as the number of unique board projects expands (as people can be traced to what social boards they regularly post on). Of course, this is dependent on how far this project expands. If it just consists of CE/RI/LUE/B8, it shouldn't be a problem, but if every social board gets its own project, I can see possible complaints.
-if the project gets REALLY popular, I could see people posting on other boards just to get in each board's database. I think I'm just being crazy with this one, but it's something that is possible.

Also, another thing that is in the back of my mind- we know that there is currently some error in the database due to manual input and whatnot (my guess is that 99% of the errors are in the low user id's). I think that this is a good opportunity to get rid of errors, depending on how you guys implement it. If an ID's name is different after being adding to a board specific database, it could automatically detect this, or a daily/weekly comparison could be made. This has actually been in the back of my mind for a while- I would think that checking both the ID and name during fetching would require a bit more time, so if error checking during fetching is not possible, it could be checked at a later time.
The Ohio State University: We are so much better than m*ch!g@n.

Edited by RockMFR at 15/03/2006 3:50:35 AM

From: Ngamer | Posted: 15/03/2006 6:56:34 AM
Wait a minute here Rock, as you suggesting that we sort of... go both ways?

Well, I've never been one to turn down a chance to play for the other team. I like the way you think.

Let's go for it.
From: Ryan Dwyer | Posted: 15/03/2006 7:06:13 AM
If this happens, users for individual boards would be fetched when the main fetch is used. There would be links on the main page to "mini-sections".

We also need to make a list of boards to do this with...

I will not use abbrev.
From: RockMFR | Posted: 15/03/2006 7:31:52 AM
I'd recommend leaving out TO. Maybe have an initial "launch" of the big 5 (ce/ri/lue/potd/b8) and then maybe in the future a launch of the medium-sized socials (to? couch? hacked?). Maybe promote it on some of the old delinked socials (ZSB, etc) which have small but dedicated and old userbases.

I'm the pitcher ;)
The Ohio State University: We are so much better than m*ch!g@n.

Edited by RockMFR at 15/03/2006 7:32:48 AM

From: Luster Soldier | Posted: 15/03/2006 9:18:16 PM
I did a lot of fetching on LUE a few weeks ago. Now, I rarely get new users from fetching LUE. Now I usually fetch topics from game boards, game social boards (Which is my current focus), and other types of boards besides LUE, Board 8, CE, RI, and PotD.

With the exception of LUE, all other board-specific user databases will keep growing over time as new users start posting on them.

As for LUE, one user (Tinister) tried to create a list of all LUEsers and it only contained about half of the current number with access.

Edited by Luster Soldier at 15/03/2006 9:21:33 PM

From: RockMFR | Posted: 15/03/2006 11:26:12 PM
I am going to have to disagree with you, Luster. I don't think they will grow. BUT MY WANG WILL!
The Ohio State University: We are so much better than m*ch!g@n.