46 coaches online • Server time: 00:54
* * * Did you know? The highest combined winnings in a single match is 250000.
Log in
Recent Forum Topics goto Post Grotty Little Tourna...goto Post The Light and The Da...goto Post TournyS(3!):Slaughte...
Christer
Online
Khemri Tomb Kings
Star
Khemri Tomb Kings
Record
59/24/37
Win Percentage
59%
Shambling Undead
Super Star
Shambling Undead
Record
51/5/10
Win Percentage
81%
Overall
[R]
Star
Overall
Record
229/56/79
Win Percentage
71%
Archive

2019

2019-04-14 23:33:08
rating 6
2019-04-07 16:59:39
rating 6
2019-04-07 00:55:26
rating 6
2019-01-08 15:27:38
rating 5.9
2019-01-05 02:58:18
rating 5.8

2018

2018-08-17 17:28:31
rating 6
2018-08-15 00:05:40
rating 6
2018-07-17 20:17:40
rating 6
2018-06-28 14:28:08
rating 5.9
2018-05-23 17:55:10
rating 6
2018-05-10 22:42:46
rating 6
2018-05-09 19:42:28
rating 6
2018-04-30 10:44:23
rating 5.8
2018-04-23 12:33:02
rating 5.8

2017

2017-04-23 18:06:35
rating 6
2017-04-06 23:00:56
rating 6
2017-04-03 19:06:00
rating 6
2017-03-29 22:35:46
rating 6
2017-03-25 16:18:39
rating 6
2017-03-11 21:24:26
rating 6
2017-02-14 14:23:58
rating 6
2017-02-10 14:54:03
rating 6

2016

2016-11-30 00:04:21
rating 6
2016-11-27 23:40:04
rating 6
2016-11-17 18:18:07
rating 6

2015

2015-09-06 23:59:26
rating 6
2015-01-24 15:56:29
rating 6
2015-01-22 13:10:32
rating 6
2015-01-19 21:20:53
rating 6
2015-01-10 19:03:45
rating 6

2014

2014-09-09 15:35:53
rating 6

2013

2013-04-26 11:48:40
rating 5.7

2012

2012-12-18 17:37:29
rating 5.9
2012-11-18 18:19:19
rating 6
2012-09-25 13:47:16
rating 5.6
2012-08-15 12:31:53
rating 5.9
2012-08-10 23:12:22
rating 5.9
2012-06-27 22:53:48
rating 5.9
2012-04-10 11:56:38
rating 5.9
2012-03-07 13:52:00
rating 5.9
2012-02-16 16:59:56
rating 5.9
2012-02-04 19:00:41
rating 5.3

2011

2011-07-25 23:32:43
rating 5.6
2011-05-23 13:12:52
rating 5.6
2011-02-04 14:26:18
rating 5.4

2010

2010-03-26 11:38:41
rating 5.1
2010-03-01 12:16:53
rating 5.6

2009

2009-12-08 16:40:30
rating 5.8

2008

2008-09-11 14:47:19
rating 4.1
2008-02-26 21:16:54
rating 5.3
2008-01-21 01:01:58
rating 5.6

2007

2007-11-06 21:23:14
rating 5.1
2007-10-16 00:26:11
rating 5.4
2007-09-30 17:10:03
rating 5.4
2007-09-30 12:01:42
rating 5.3
2007-08-09 12:14:57
rating 4.5
2007-08-06 12:02:52
rating 4.9
2007-08-03 17:56:21
rating 5.4
2013-04-26 11:48:40
44 votes, rating 5.7
Building a game finder
Over the last couple of days, the game finder on FUMBBL has been acting a bit erratic. I was going to write this in a forum response, but decided that it would be better to post as a blog entry.

The core functionality of the game finder is actually not that hard to explain. Simplifying a bit, the system does the following:

1. Track which teams are entered into the system.
2. Find matches that are valid.
3. Show these matches to coaches in a list
4. Track offers (ie, the accept/decline status) for each match
5. Start matches which have been accepted

To actually make this happen, though, is much much harder. FUMBBL is mainly written in the PHP language, which is really great for certain things, but one of the major limitations of it is that it is very hard to maintain a persistant "state". This means that it's hard to keep track of things between each page load. The game finder system, for example, needs to keep a list of potential matches and the accept/decline state of each match. While this could be done in a database, it would end up being pretty slow when there are many people active in the game finder.

When I made the decision to build the game finder in the first place I decided to build it in Java, which was partly possible because of the upgraded web server (ie, I added a fair amount of RAM and could afford to set up a Java EE server). Having the Java server up and running means that I can utilize a persistent state and keep certain things in the game finder in memory.

Some time ago, when I built the blackbox scheduler (also in Java, but in SE rather than EE), I developed a database abstraction layer to simplify the access to the FUMBBL database from Java. This is a relatively complex task in itself and involved a number of subcomponents. The most important one in this context is a cache, which allows the system to not be forced to make database queries every time the application needs data. Without this cache and abstraction layer, code would look somewhat like this (very much simplified):

resultSet = database.query(TEAM_SQL, teamId)
teamName = resultSet.firstResult.get("name")
teamTV = resultSet.firstResult.get("tv")

display(teamName, teamTv)
... etc


In real code, this would probably be moved over to some kind of class though:

class Team {
var name
var tv

method load(teamId) {
resultSet = database.query(TEAM_SQL, teamId)
name = resultSet.firstResult.get("name")
tv = resultSet.firstResult.get("tv")
}
}

team = Team.load(teamId)
display(team.name, team.tv)


While this method is better, it's still wasteful of resources since every time you want to access team information, you'd need to load the team data from the database. To solve this problem, you add a cache. Simply put, a cache is a place where you can store things for later use. With a cache, the code would become something like this:

class Team {
var name
var tv

method load(teamId) {
if (cache.getTeam(self, teamId)) {
done;
}

resultSet = database.query(TEAM_SQL, teamId)
name = resultSet.firstResult.get("name")
tv = resultSet.firstResult.get("tv")

cache.storeTeam(self, teamId)
}
}

team = Team.load(teamId)
display(team.name, team.tv)


The main difference here is that when loading a team, the code checks if the team was already stored and if so, fetches the information from the cache rather than reading it (again) from the database. Of course, there is much more to caching than this but I'm simply trying to illustrate the core concept.

Although this is greatly simplified, the game finder is using a system that is similar to this. One of the major problems with caching in general is that if you have multiple systems (for example, the FUMBBL PHP website) that can modify data, caching can lead to displaying outdated information. For example, if the Java application has the team information in the cache (in memory) and the team name would be changed in the database, the Java application would not see this change. If you remember the early days of the new game finder, this was the case for some teams where the TV seen in the game finder wasn't the TV seen on the website. Basically, an uploaded result updated the database, but since the team was in the Java cache, the old TV was used for game finding. The way this was solved was to make the PHP site send a message to the Java application to "flush" (ie, remove) the specific team object from the cache when PHP code changed data on the site (of course, this is more complex than it sounds).

To get back to the game finder itself, the way that it functions is that it keeps a list of teams, and every time a request is made from someone showing the game finder view is that it searches through all possible matchups and gives back a list of them, together with their state (the accept/decline status of the match). There are a few of problems with this method:

1. This is very time consuming. Checking each possible match is a slow process, where the code checks each possible opponent for each team in the system (this is called O(n^2) complexity) and this will slow down exponentially as the number of teams increase.
2. Since the method is called every time a list of matches is requested (once every 4 seconds when you're on the game finder page), this slows down the web server pretty quickly. I've found that the Java process that does this is making page loads on the site slow down.
3. Keeping the list of teams in memory creates similar problems as with caching. The team objects are not released at any point, and the cache isn't given an opportunity to refresh outdated data.

The third point, together with a similar issue with a "coach" object stores a list of teams, is creating the problems that people are seeing with the game finder now. The reason it is a bit more broken than it was last week is that I made an attempt to fix the slowness by improving how things are processed (which highlighted the core problem to a greater extent).

In order to fix these problems, I will need to do a relatevly significant rewrite of the game finder. A couple of things I will be doing is:

1. Decouple the game finder lists from the actual abstract objects. Instead of storing a list of Team objects, I will store a list of team IDs, and repeatedly ask the cache object for the teams when needed. This will allow the cache to properly refresh team information as appropriate.
2. Maintain a list of possible matches properly, and update this properly when a team is added, removed or changed. This will improve the performance of the system, and the site as a whole since teams changing is significantly less frequent than requests for possible matches.
3. Properly isolate the list of teams from the "framework" code for the game finder. Currently, the code is somewhat monolithic with too much logic running in the same code context. Doing this will improve the readability and maintainability of the code.

Since this is having a relatively big impact on the site at the moment, I am hoping to be able to find the time to sit down and work on this this weekend. Until then, I am thanking you for your patience and apologize for the inconvenience of the game finder not working as it should.
Rate this entry
Comments
Posted by Dalfort on 2013-04-26 12:01:06
Would a limit on the number of teams a coach enters to the game finder help significantly, or does this go against your ideals of the system as you visualise it?

and...

Thanks again for your commitment and tireless work maintaining and improving the site which we all enjoy so much (that sounds far too sycophantic but it is genuinely meant!).
Posted by Chainsaw on 2013-04-26 12:01:10
I'm a fan of Java. :)

Although, FYI for non-coders, the snippets in this blog are JavaScript (quasi code).
Posted by maysrill on 2013-04-26 12:44:13
Would a (temporary?) change in the check time from 4 second to something like 15 be of much help? I know "instant" is nice and everyone likes it, but the system would be doing about 1/4 the work for very little change in functionality to the end user.

BTW, I had a very nice, old-school IRC challenge last night and got to see on-pitch logos for the first time in a while. Had a fun game out of it. I'm not suffering here ;)
Posted by blader4411 on 2013-04-26 13:48:57
Does the method of starting a game determine the viability of pitch logos? I thought it was because some OS (i.e. Mac OS X) mirror the Java overlay leading to vanishing logo and gibberish team names.
Posted by RC on 2013-04-26 13:52:05
I didnt understand anything but I am very appreciative of all your hard work. :)
Thank you!
Posted by maysrill on 2013-04-26 15:02:57
@blader - If you start a game using a game name (i.e. arranged in IRC, scheduled tournament game, etc.), you get logos. If you use one of the schedulers (BBox or GF), you don't. I don't know the mechanism involved, but it's a low-priority bug that popped up from one of the client updates.
Posted by baelnic on 2013-04-26 15:47:12
So my suspicions are true. The Chinese government has hacked fumbbl in an attempt to steal our fantasy football secrets!
Posted by Hitonagashi on 2013-04-26 17:13:35
Oi Chainsaw, it's just pseudo code. Javascript is not quasi-code, it's an awesome language. (it also doesn't have a 'class' declaration without including frameworks/compilers).

Christer, can't it be improved from n^2? If you keep an array consisting of a hashmap of team id -> low_bound and team-id -> high_bound, then the possible matches are the intersection of the sets where the TV is greater than the low bound and less than the high bound. This involves a) iterating through low, b) iterating through high, and c) iterating through a and b to find the intersection, which is 4n? You then recalculate the overall low and high bound whenever a team is inserted into the game finder.

I think that's pretty much what you are proposing though, don't know how much of an improvement that would offer when you move to calculating matches on insertion.
Posted by Hitonagashi on 2013-04-26 17:13:56
P.S, loved the post, forgot to mention. Really good read thanks!
Posted by uuni on 2013-04-26 17:31:36
Could the situation benefit from moving some parts of the pairing to the clients with JavaScript etc?

If each n clients would perform something like:
a1. fetch all-Team list
a2. fetch own-Team list
a3. filter matches locally, O(all x own) or perhaps less

And then, when user changes a state of a match own-all1:
b1. Send statechange(t1 from own, t2 from all) to server

Which would not be recorded on the server but instead just propagated to the other end:
c1. Send statechange(t2,t1) to t2-owner

This would change the server to require active sessions to each client, but I think it sort of already does something that is similarly resource intesive.

I think this could benefit the system by moving the bulk of the load from the server to each game client. As the amount of client scales linearly with itself, it could be of use.

***

Of course, this could be just some thought foam as I am quite bit fluish right now. Hope it was of some use! :-)
Posted by Kelkka on 2013-04-26 17:49:40
I know this solution is unacceptable, but you could distribute the stress from server side to clients. Make it with javascript, browser polls the complete list of teams and data bound to them.

And browser is then responsible from handling the data and forming the list of games. This would increase data in/out quite a bit, but would release a lot of memory and cpu time from the server.

Downside is extra coding, handling the communication over net, obvious security risks and exposing all game finder information to all of the users.
Posted by harvestmouse on 2013-04-26 20:44:12
@baelnic

Looking at the Chinese release notes for next month I saw:

Harry Potter 57
One Piece movie 19
Fantasy Football version 6
& Servant to the Duke of Rings (Return of the Queen)

Personally, and without wishing to make waves on the new gamefinder. Finding a game without IRC (working) not being able to see what else is on gamefinder not working (to the point of costing FUMBBL a lot of unplayed games).
Posted by Urrghs on 2013-04-27 04:10:22
How dare you not to give us a perfect gamefinder immediately??

Lol; honestly mate, you give me great laughs anytime you apologise to us the community about something. Really; where would we be without all of your efforts? Well; maybe some more of us would have girlfriends or wives and would have finished work/thesises etc. earlier, but still....where would be the fun? (and blood).

So; thanks a lot.