| Poll |
| Which is better? |
| Cherry Pie |
|
26% |
[ 18 ] |
| Apple Pie |
|
43% |
[ 29 ] |
| Bakewell Tart |
|
29% |
[ 20 ] |
|
| Total Votes : 67 |
|
harvestmouse
Joined: May 13, 2007
|
 Posted:
Nov 11, 2014 - 16:02 |
 
|
Ahhh yes, that's correct, the previous version didn't. I don't think you 'should' be allowed to blitz. However, unfortunately the rules disagree. |
|
|
King_Ghidra
Joined: Sep 14, 2009
|
| Posted:
Nov 11, 2014 - 16:12 |
|
| harvestmouse wrote: | | Just 9 games.......well........so if complaints start coming in about bugs in the client we know who to blame. Every single person who hasn't been playing in test mode! |
| Kalimar wrote: | Perhaps I need to explain the purpose of such a public Beta a bit better.
A multi-user client-server software like FFB is difficult to test in all its aspects. A lot of bugs appear only in very specific situations under stress. Several games running simultaneously, lots of spectating, some real load on the server. How is the performance then? How stable is it in this situation? Where is the limit?
Every strange behaviour we can find in the Beta, every bug that gets detected and fixed will make the transition to the "live" system smoother. Every server crash we get in Beta is one we can possibly prohibit from happening when the "real" divisions run on the system. We'll know what to look for...
To be able to say anything meaningful about the quality of the new code we'll need at least 100 games or so. The more the better. Even better with several games running in parallel.
It's in your hands, really... So do something about it! |
For my sins, I run a QA department as a job, so this kind of thing isn't my idea of leisure, but let me make some points and ask some questions.
First - the specific changelist. What/where is it? At the moment it is a little vague (I've read the blogs, but they contain lines like "a lot of bug fixes". If you want focused and targeted testing on changes rather than simply people playing the game and finding nothing, then it would be better to emphasise to those willing testers what the areas to focus on are.
Perhaps the top post in the thread could contain a breakdown of functional and technical changes and perhaps a checklist of areas which have and have not had significant attention from the community?
Secondly, the non-functional stuff - just having your fingers crossed isn't going to get any load or stress on the server. Wouldn't it be easier to use a dev/testing tool to simulate these kind of situations? What are the actual expectations / benchmarks /scenarios for server interaction and load?
Finally a general comment - fumbbl might be a community-driven site in many ways, but the client and server development is totally opaque to the users. They come to here enjoy themselves - and perhaps some number of them give up their hard-earned money in donation with no other expectation in return than maybe the site still exists next time they visit. To imply that potentially game-breaking updates are their fault or something they have no right to complain about is completely wrong imho. No one should be making any changes unless they are completely confident in them, and if that means the test division runs for another six months or those on the dev side have to find more creative ways of testing this themselves then so be it. Users can help, sure - and the powers that be should be grateful for anything they get. |
|
|
Kalimar
Joined: Sep 22, 2006
|
| Posted:
Nov 11, 2014 - 16:45 |
|
As I feel King Ghidra's comment is directed at me I'll try to answer in full (this is going to be a somewhat longer post):
1) The purpose of this Beta Test is quite simply to play games start to finish. No more. No less. The more of them the better. If there are obvious bugs found I'd be happy if they are reported to the right channels (Help -> Bug, Category FFB Test). More important to me however, is to check the tons of server logs written in the process and find problems in the FFB - Fumbbl integration.
2) The coaches playing in this Beta should simply enjoy themselves and not necessarily look for specific changes and/or bugfixes. We had a closed testing phase for that a while back and it's much more efficient to run the client in testmode without uploading the game (where you have chatcommands to set skills, modify stats, roll dice, etc.). Uploading games on the other hand can only be properly tested with actually finished games (something testers looking for a specific bugfix rarely do). I can't for instance say much about the average size of an uploaded replay without having some "normal" games finished.
| King_Ghidra wrote: | | They come to here enjoy themselves - and perhaps some number of them give up their hard-earned money in donation with no other expectation in return than maybe the site still exists next time they visit. To imply that potentially game-breaking updates are their fault or something they have no right to complain about is completely wrong imho. No one should be making any changes unless they are completely confident in them, and if that means the test division runs for another six months or those on the dev side have to find more creative ways of testing this themselves then so be it. Users can help, sure - and the powers that be should be grateful for anything they get. |
3) I do software development for a living myself and have been in a QA role as well as leading a team of developers. So - like yourself - I don't really enjoy doing quite the same in my free time. This is the main reason the code is not simply in the public domain. This would turn my role from a developer to project lead. And I - quite simply - don't want to be that for a "fun" project. I AM grateful for any test plays done by the users. I simply tried to motivate everybody to play more games. Any "blame" posts didn't come from me, as you might have noticed. Codewise this is a one-man-show. And this one man can put as much effort in as he has time (family and job come first) and motivation at this particular point in time. Just like that.
4) The suggestions made all make good sense. However I don't have time or equipment for an automated load test. Having a script actual simulate gameplay is a rather tough and time-consuming task (in larger companies there is an exra department for tasks like that). Coaches play all over the world with lag, connection failures, etc. all coming into play. Theoretically I'd know what to do to cover this. But then I'd have to work on this with a team and a paid 8h workday. Not hacking away 2h at night while watching the news on tv  |
|
|
koadah
Joined: Mar 30, 2005
|
| Posted:
Nov 11, 2014 - 17:16 |
|
|
Christer
Joined: Aug 02, 2003
|
| King_Ghidra wrote: |
For my sins, I run a QA department as a job, so this kind of thing isn't my idea of leisure, but let me make some points and ask some questions. |
Ok, I'll respond to this as well. First off, let me be direct and state that I was initially offended by the way you write this post. In my reading of it, it comes off as a post where you feel that users are entitled to some particular level of service, and that the developers of FFB and FUMBBL (ie, Kalimar and myself respectively) are here to work for you guys.
However, I will let that be as I expect that this isn't the way you intended the post to read and call that an effect of this being in written form.
So, as I feel the topic pertains to me as well, being the sole developer for the website (and contributor to the FFB project).
I normally try to not do full quotes in a post, but feel that I should in this case. I don't want to leave information out.
| King_Ghidra wrote: |
First - the specific changelist. What/where is it? At the moment it is a little vague (I've read the blogs, but they contain lines like "a lot of bug fixes". If you want focused and targeted testing on changes rather than simply people playing the game and finding nothing, then it would be better to emphasise to those willing testers what the areas to focus on are.
Perhaps the top post in the thread could contain a breakdown of functional and technical changes and perhaps a checklist of areas which have and have not had significant attention from the community?
|
To me, a full change log isn't exactly practical. It would either be a list of actual commit messages (which tend to be pretty technical at times and not useful to end users). Both me and Kalimar run version control, and if I were to look back at the commit messages I write for FUMBBL, they're not very descriptive in general. In my case, there are also commits that are done for future features, which I don't particularly feel a need to announce in a public fashion. It's not as if they're secret, but I would rather not have people hammering on parts of the website I don't feel are ready to announce (and yes, I use the live site/db for testing purposes at times).
So, let's say we can agree that full version control logs won't be a good fit here. That leaves a curated version of the log (pruning/rewriting messages, based on the actual list of commits), which I agree would be nice. However, the problem is that someone needs to do this work, which could at times be quite time-consuming. Mapping a commit to the features it may affect in a given project can be a pretty daunting task.
I don't make changes I do on FUMBBL public for this reason. If I had to maintain a public change log, it would add even further barriers to actually making changes in the first place. Developing the website is, as is the case for the FFB client/server, a one-man operation. Would it be possible to take in more devs? Maybe, but finding the right person to do that isn't exactly easy.
| King_Ghidra wrote: |
Secondly, the non-functional stuff - just having your fingers crossed isn't going to get any load or stress on the server. Wouldn't it be easier to use a dev/testing tool to simulate these kind of situations? What are the actual expectations / benchmarks /scenarios for server interaction and load?
|
A simulation is only ever going to be that - a simulation. While it's certainly useful, you again have to weigh the cost (time/resources) of performing simulations to the benefits. In single-developer projects, having a toolset for simulation is quite costly (ie, time consuming). For FFB, I'd have a hard time finding a tool that could do a realistic simulation in the first place. For FUMBBL, there are certainly tools that simulate web users, but I'd end up having to spend months finding usage patterns to make a realistic simulation, and it would be completely useless for new features to the site.
So my answer is that I don't believe it would be easier to use a testing tool. It's simply too hard to set it up (and maintaining it as the codebase changes).
In terms of actual resource usage, the FFB server is incredibly light-weight. The hardware it runs on (the relatively new Roxanna server) could easily handle hundreds of simultaneous games. As for the website, it could easily double the traffic without breaking too much of a sweat.
| King_Ghidra wrote: |
Finally a general comment - fumbbl might be a community-driven site in many ways, but the client and server development is totally opaque to the users. They come to here enjoy themselves - and perhaps some number of them give up their hard-earned money in donation with no other expectation in return than maybe the site still exists next time they visit. To imply that potentially game-breaking updates are their fault or something they have no right to complain about is completely wrong imho. No one should be making any changes unless they are completely confident in them, and if that means the test division runs for another six months or those on the dev side have to find more creative ways of testing this themselves then so be it. Users can help, sure - and the powers that be should be grateful for anything they get. |
Ok, bringing up donations is a low blow and doesn't belong in this discussion. Let me respond to that real quick though. The donations are there in order to offset my direct costs of keeping the site up and running. The way things are set up (servers running in my home), the whole operation runs on a shoe-string budget ($100 per month running costs, ie power, bandwidth, offsite backups, domain registry and various certificates, and roughly the same per month for hardware updates). The donations do not cover the actual costs of this, and I end up "donating" to the site to cover costs pretty much every month. I don't have a huge problem with this, and I do donation drives to cover hardware replacement/upgrades. This works out pretty well, and allows me to run the site without ads. But, donations have nothing to do with server/client development, nor do I charge for the time I spend developing the site (I ran a code analysis which said the custom code for the website was equivalent to 10 man years more or less; you do the math on what that would be per month). It's not really fair to bring it up in a discussion like this, because it's quite frankly a slap in the face and irrelevant.
Kalimar has at no point tried to blame coaches for bugs in FFB, and the same applies for me for bugs on the website. Bugs happen, and in particular on the website because I end up doing things in less than optimal ways at times. I take full blame for these bugs, and I try to remember to apologize to people who get affected by my mistakes. Do I feel "completely confident" about my updates? No, I don't. I won't ever be, because I've been developing for long enough to realize that coding is really really hard, and there are so many edge-cases and subtle misunderstandings of how things work. Perhaps that's a bit cynical, but for the website I've chosen to be prepared to revisit things and fix bugs quickly rather than spend weeks testing on my own before I push things live. The end-users are on such a wide variety of platforms that I simply don't feel it's worth the effort to do full testing for every change I do.
You can call me a proponent of continuous integration, or perhaps agile methodology (although that has some other meanings in a developer sense; I don't do agile development with sprints and whatnot). It might not be the most professional way to do things, but from my perspective, FUMBBL isn't an enterprise product. It's effectively my hobby which allows me to do fun development stuff mostly on my own terms without the formalities and budget constraints that I spend time on at work (I work as a developer in the finance industry).
Finally, your closing comment about us being grateful for anything we get. Yes, and I assure you we are. I put tremendous pressure on myself to try to do the "right thing" when it comes to the site. I could technically do whatever on this site, including all sorts of unfair advantages for myself and the staff of volunteers who help out supporting coaches on the site. However, I try to keep this place as good as possible for the whole community which includes keeping things fair. Back when I started the site, it wasn't really meant to become what it is today. I simply wanted to play blood bowl with the best client that was available at the time. That is still the case, even though I rarely take the time to actually play these days.
My point here, though, is that me and Kalimar aren't demanding anything, but you have to realize that this goes both ways. To make something like FUMBBL or the FFB Server/client, there's a huge investment in time and effort. Both me and Kalimar have put thousands of hours (literally) into our respective projects, and it can be frustrating for us to reach out and request assistance to test things and not be getting a response from users. Because in the end, pushing that codebase live will always result in people experiencing bugs, and end up disappointed when it's something that can't be fixed and they lose their partial game (effectively wasting an hour of their time, or changing the outcome of a tournament). This type of thing is also tough from a developer perspective, as we inevitably blame ourselves for the bugs and issues.
In the end, it's inevitable that we get negative feedback from pretty much everything we do. This is obviously offset by a number of people who are happy for changes, and expecially the few who express gratitude. We honestly try our best with the resources and energy we have. With the site and the FFB client/server being non-commercial projects, this means that we cut corners to make changes and that we're able to do things that are simply not feasible in an enterprise environment (complete rewrites of the networking subsystem in Kalimar's case for example).
Bottom line here is that we value feedback, and appreciate those of you who spend your time testing things. Testing your own code is usually very hard as it's tough to identify edge conditions you haven't thought about while writing the code (or in other words, it's hard to think of things you didn't think of).
Phew, long post. Sorry about that.  |
|
|
Dominik
Joined: Oct 29, 2004
|
| Posted:
Nov 11, 2014 - 19:42 |
|
I'd play more test games if it is assured that those games do not cound - at least permanently - for the coach statistics. |
|
|
CCSenor
Joined: Aug 15, 2009
|
| Posted:
Nov 11, 2014 - 19:53 |
|
I'm sure i read somewhere that they don't count Dominik, but it really shouldn't matter anyway . . . |
|
|
Garion
Joined: Aug 19, 2009
|
| Posted:
Nov 11, 2014 - 20:00 |
|
| Dominik wrote: | | I'd play more test games if it is assured that those games do not cound - at least permanently - for the coach statistics. |
they don't count, and really why would you care even if they did?
But they don't. |
_________________
 |
|
Dunenzed
Joined: Oct 28, 2011
|
| Posted:
Nov 11, 2014 - 20:02 |
|
| Dominik wrote: | | I'd play more test games if it is assured that those games do not cound - at least permanently - for the coach statistics. |
Dude, are you ok? Maybe it's time to step back for a bit if the most important thing is your coach statistics. Elves, Dwarves, Vampires, Goblins, Wizards, Chainsaws. These are some of your clues that this isn't a serious thing.
To stress about a thing that is largely trivial and fairly easy to manipulate doesn't seem like a healthly place to be. |
|
|
koadah
Joined: Mar 30, 2005
|
| Posted:
Nov 11, 2014 - 20:21 |
|
| Dunenzed wrote: | | Dominik wrote: | | I'd play more test games if it is assured that those games do not cound - at least permanently - for the coach statistics. |
Dude, are you ok? Maybe it's time to step back for a bit if the most important thing is your coach statistics. Elves, Dwarves, Vampires, Goblins, Wizards, Chainsaws. These are some of your clues that this isn't a serious thing.
To stress about a thing that is largely trivial and fairly easy to manipulate doesn't seem like a healthly place to be. |
This is real dude!
Each one of those players represents a real being in a parallel universe.
If you don't look after your players you will be looking at a terrible after-life! |
_________________
New teams. Secret League or Official. Always recruiting! |
|
harvestmouse
Joined: May 13, 2007
|
| Posted:
Nov 11, 2014 - 22:35 |
|
| King_Ghidra wrote: | | harvestmouse wrote: | | Just 9 games.......well........so if complaints start coming in about bugs in the client we know who to blame. Every single person who hasn't been playing in test mode! |
They come to here enjoy themselves - and perhaps some number of them give up their hard-earned money in donation with no other expectation in return than maybe the site still exists next time they visit. To imply that potentially game-breaking updates are their fault or something they have no right to complain about is completely wrong imho. No one should be making any changes unless they are completely confident in them, and if that means the test division runs for another six months or those on the dev side have to find more creative ways of testing this themselves then so be it. Users can help, sure - and the powers that be should be grateful for anything they get. |
Wow really! Although the powers that be maybe grateful and don't expect anything, my view is different.
If you don't have the time to play, that's fine. However if you do play, and you don't have a fixed game I believe it's amazingly selfish to play in one of the main divisions rather than give something back to the site and play a game in the Test division. It's not like it's work even. Most test games are just like any other game.
The test team have already done the majority of the work. This is a polish job. It's great fun.
My view is totally different to yours. I don't think Christer or Kalimar should feel grateful for almost anything. If you calculate time and money invested, they come up short big time. How many games do you see either of them getting to play? Not that many, not many at all.
Nobody implied that game breaking updates would be implemented. Your pushing a snowball into an avalanche there. However I will say that testing in the Test Division is very simple thing and it's just as much fun as playing in a 'competitive' division. I honestly feel those that are playing a substantial amount of games that aren't playing in the Test Division are extremely selfish. And if bugs do get through testing, and they suffer from them. I have zero sympathy at all.
This maybe Christer's site. We may have to play by his rules. However it is a fan site and a free to play site. I think too many take that for granted sometimes.
It's not much, come on guys do your bit. I'd be surprised if anything more than a normal game happened in most cases..... |
|
|
harvestmouse
Joined: May 13, 2007
|
| Posted:
Nov 11, 2014 - 22:36 |
|
| Dunenzed wrote: | | Dominik wrote: | | I'd play more test games if it is assured that those games do not cound - at least permanently - for the coach statistics. |
Dude, are you ok? Maybe it's time to step back for a bit if the most important thing is your coach statistics. Elves, Dwarves, Vampires, Goblins, Wizards, Chainsaws. These are some of your clues that this isn't a serious thing.
To stress about a thing that is largely trivial and fairly easy to manipulate doesn't seem like a healthly place to be. |
Dominik's special. You won't be able to reason with him. Just leave it be, it's not an issue. |
|
|
mrt1212
Joined: Feb 26, 2013
|
| Posted:
Nov 11, 2014 - 22:48 |
|
Has the hmp bomb eating been tested thoroughly? |
|
|
harvestmouse
Joined: May 13, 2007
|
| Posted:
Nov 11, 2014 - 23:04 |
|
| mrt1212 wrote: | | Has the hmp bomb eating been tested thoroughly? |
I don't know, have you tested it? That's the point. What testing have you done on it?
You have played 22 games since open testing began. How much of a concern is it really, if you haven't bothered to play a small percentage in the Test Division? |
|
|
WhatBall
Joined: Aug 21, 2008
|
| Posted:
Nov 11, 2014 - 23:05 |
|
If people want to test some crazy stuff, like a lot of HMP with no Secret Weapon, past links to your players here with skills you want and I will see what I can do. Can't promise I can get to them all, but I will do what I can. |
_________________
 |
|
|
|
| |
Draft League Expansi...
Search
