zondag 20 februari 2011

My 'Belgium Testing Days 2011' experience

be prepared.. this is a loooooong blogpost :-)

An easygoing startup...

On Monday afternoon I embarked the Thalys to bring me to Brussels. I experienced a slight delay and the train didn't go any further then Brussels (midi), which was unfortunate for all travelers to Paris, but I was lucky because I was at least (almost) at my destination. After a short travel with the local train to Brussels Airport I arrived at the Sheraton where the Belgium Testing Days were to be held on Tuesday and Wednesday. This is my first time that I'm actually staying at the hotel which is at the same location as the conference, which is very convenient I experienced.
The room was okay; although I found that an 'open wire' on my bedside lamp was quite dangerous and the TV didn't start up because of a resolution error (- always a tester I guess, checking the room :-) )
I found a very nice gesture on the desk from the Belgium Testing Days people: a small card bidding me welcome, a small box with Belgium Chocolates and a small bag with three different kinds of 'Jenever' (Gin).



That evening a dinner party was thrown at a restaurant in the center of Brussels. As it was Valentine's day we were a bit of weird company situated in the middle of the restaurant on a large table, surrounded by all these little 2 persons tables with couples. The three course dinner was excellent, the main course was ridiculously big though! I think my entrecôte alone was more than 600 grams and then there was veggies and potatoes there too.
Remco Oostelaar and John Bertens added a nice Valentine touch, by buying a whole bush of roses from a street merchant and handing each lady in the party one.
After a (bit abrupt) end of the dinner we were brought back to the hotel, I drank a hot choco in the hotel bar and went to bed. A large meal is truly exhausting!

An invigorating day one!

Tuesday, the first day of the conference, started at 08.30. José M. Díaz Delgado opened the conference and after that Mieke Gevers introduced the first keynote which was that of Johanna Rothman: "Lessons learned from 20 years of Managing Testing".




These are the tweets related to that keynote (deleted RT's):

  • lol at @johannarothman's tale of getting her English degree and why she decided to get a CS degree #btd11
  • http://yfrog.com/h4zyfoj : lessons learned from many years of managing testing by @JohannaRothman #btd11
  • Great stories and food-for-thought in this track by @JohannaRothman with much embedded fun! #BTD11
  • Not a new msg from @johannarothman, we can't "test quality in", but still an important message - we're info providers #btd11
  • "Software testing has everything to do with Product Information". .. Quote from @JohannaRothman 's keynote #BTD11
  • We need more diversity in testing since problems are more complex, @johannarothman #btd11 "the stuff we do is really hard!"
  • ...#BTD11 : more information means more testing approaches
  • The stuff we tested back in the 70's... By @JohannaRothman Wasn't aware testing existed before I did ;-) #BTD11
  • Excellent idea for project managers by @JohannaRothman Use weather reports instead of traffic lights or smileys! #BTD11
  • .@johannarothman reports that weather reporting is better than traffic lights. That's cloud reporting I think #btd #btd11
  • #btd11 it's more accurate to use weather report status reports than traffic light status report : ever reported a 'green light' that..1/2
  • #btd11 ..was truly honest? Honest is yellow, but that's meaningless, actually it's always red..
  • If developers look at garbage all day, they create garbage. Interesting though from @JohannaRothman #BTD11
  • Because we provide information, testers have /significant/ power in he organization. @johannarothman #btd11
  • Good advice - end the multitasking. @johannarothman #btd11
  • #btd11 : saying "NO" is necessary tool
  • @JohannaRothman says End the multitasking.Maybe for us,but a completely new multitask enabled generation is growing up at the moment #BTD11
  • Fun exercise to say "No" or "Yes" in @johannarothman's keynote (and get used to exercises in keynotes, I'll do them tomorrow) #btd11
  • @johnbertens They're not multitasking enabled, but they may have a lower cost of switching (but not necessarily) #BTD11
  • Nice to see that @johannarothman refers to congruent communication based upon Virginia Satir. #BTD11
  • I don't know when quality became software test.- @johannarothman #BTD11 +1 from my side.
  • Like @johannarothman I prefer to be called a "software tester" than a "s/w quality engineer" #btd11
  • I wonder what @johannarothman says about "Quality Assurance engineer". #BTD11
    .@financialagile for me, engineers are people who took a PE exam, & s/w is more creative/craft than engineering. #btd11

It was an invigorating start of the day. Johanna really has a way to put the energy into her audience. I liked the idea of 'losing the traffic light' metaphor and introducing the weather report status overview. Is certainly more detailed, honest and accurate. I was astonished by the effect of saying NO and YES to people while walking trough the room, it gave me a bit eerie feeling on how large an effect only a word can have.

After the non-scheduled, short break, I went to "Have you met the CAT?" by Jana Noack (iSQI) and Arjan Brands (Diaz & Hilterscheid). Being involved at training and certification at my employer, I'm alway interested in developments in this area. CAT stands for 'Certified Agile Tester' (http://www.agile-tester.org/). Alas there was a very small crowd. I think 'agile' and 'certification' are a combination that doesn't appeal to people; especially when Lloyd Roden is doing a track (last-minute fill in for Miguel Lopez who just didn't show up without notice) and a 'best practices' track (Tim A. Majchrzak) is held at the same time. The content of the 'CAT' track was however very interesting. The course seems very well thought off, good build up etc. The certification is not 'just another nice paper on the wall', the emphasis is on the content of the course, the certification is (part of) the evaluation at the end. Again it was expressed that the certification doesn't make you an Agile Tester, it's the experience and practice that does. I know the content will probably be under heavy debate within the testing community, but actually I think that this is a good starting point for whomever gets involved in Agile projects.



Again a break after this (a schedule one :-) ). Great coffee here and small pastries to go with it. The expo itself isn't that large, plenty of opportunity to visit all vendors during the different breaks. Breaks always seem short during conferences, there's always some interesting conversation to be involved in and time flies by during those talks. It's always so much fun to see all these discussions amongst the audience regarding the track they just visited, adding their own ideas, agreeing, disagreeing. This is what makes a conference truly worth visiting!

After the break I went to Susan Windsor's track about 'How to create Good Testers'. I'm a bit of a fan of Susan's talks, I chaired her workshop at EuroSTAR 2009 and have been to every track of her whenever she was scheduled on the program. I always pick something up, learn something, I think the tracks have great value.

Tweets related to Susan Windor's track:

  • Now at Susan Windsor's track "How to Create Good Testers" http://yfrog.com/h2emzlvj #BTD11
  • enjoying Susan Windsor's pragmatic approach and good practices, good testers at #BTD11
  • #btd11 : "if 2 people agree on everything all the time, you actually only need 1 of those 2" -thought -Susan Windsor. - difference is good
  • Susan Windsor sez u need domain space expertise to be valuable as a functional tester, also internal network in org #BTD11
  • Susan Windsor: take the responsiblity to improve, take responsibility for your own career... #BTD11
  • Susan Windsor: performance tester: scripting is *min* reqt; understanding arch is main reqt #BTD11
  • Susan Windsor: Test mgr=PM+test+people+test+troubled projects+high bs detector in many disciplines #BTD11
  • Susan Windsor: annoys her when recruiters/hiring mgrs look for certs instead of experience. +1 from me! #BTD11
  • Susan Windsor: certification may not be used as a meaningful measure... #BTD11
  • Susan Windsor: Don't slavishly follow an industry guru. Think!! +1 from me :-) #BTD11
  • Susan Windsor: Thinking matters whether transferring within or hiring from outside. #BTD11
  • Susan Windsor: You are responsible for your own career devt. #BTD11
  • And another slide from Susan Windsor: http://yfrog.com/h07rohyj #BTD11 -enjoy!
  • Susan Windsor: to be a guru is not about reading books, it's about DOING things... #BTD11
  • Want to retain your personnel: tip: recognize contributing value of person :-) I
  • agree! -#BTD11 Susan Windsor
  • @jostaerwe tho reading books is good too - I know too many testers who don't even do that! #btd11
  • @jostaerwe I agree and it's no coincidence that all guru's and rich people read tons of books #btd11



LUNCHTIME! There was a buffet (which had been hiding behind large doors in the expo space) with all kinds of colds, hots and a VERY appealing dessert table. The desserts actually looked THAT tasty that I was seriously considering skipping the healthy stuff and only dig into the sweets. But I was 'sensible', grabbed some salad and some very good stew and did some selected sweet hunting after that.

The next track I went to was that of Remco Oostelaar and John Bertens, they are my colleagues from Capgemini and - besides that their track appealed to me- I wanted to support them in their first international performance together. Remco and John set up their track as a 'play', impersonating two testers (each a specific rol of 'old skool tester' and 'next gen tester') from the 'Bank of Angels' which is about to start with Cloud services. Their discussion leads the audience through the cloud material and the discussions that arise regarding 'V-model', 'waterfall' and 'agile', on-the-fly they show an arsenal of Cloud tools to be used in testing.

Tweets regarding this track:

  • In this #btd11 session on cloud computing & agile we're encouraged to tweet! Nice multi-media preso already!
  • COO wants a test approach that will let business really sit in the driver's seat. How can we do that? #btd11
  • Love the role playing approach for this preso - very effective! #btd11 true to life!
  • now at: @RemcoOostelaar & @johnbertens track #btd11 "the business in the driver seat with cloud computing" - a dialogue: having fun!
  • Cloud offers multiple levels of services - infrastructure, data, software (and one I didn't catch) #btd11
  • Twitter is a cloud service. Other examples, SalesForce, GoogleDocs, MS Azure, Force.com, Google apps.... #btd11
  • Cloud advantages: flexibility, mobility, freedom to focus on innovation, less cost, don't need to worry about updates #btd11
  • Thanks @FunTESTic now I have twitter ids for @johnbertens and @RemcoOostelaar, loving their preso so far #btd11
  • We need to get out of the testing box in the old waterfall model @RemcoOostelaar @johnbertens #btd11 Oooh, more mindmaps!
  • Enjoyed realtime poll from @RemcoOostelaar and @johnbertens #btd11 Cloud wins!
  • It's a tough way to present this way: but Remco and John seem to pull it off! Having good time...#btd11
  • It's funny, I think of the "cloud" as something scary that I don't know about, but actually it's something we use daily #btd11
  • Cloud tools focus on collaboration between teams...#btd11 john&remco
  • Use the cloud for networking, communication, collaborating, sharing @RemcoOostelaar @johnbertens #btd11 connect biz/tech team
  • hmmm, the tool they are showing looks heavyweight for an agile project - but maybe looks are deciving #btd11
  • Personally I still prefer post-its on paper to the cloud tool. We can take photos of the post-its on paper 4 remote ppl IME. #btd11
  • External testing - SOASTA, LoadStorm, Browsermob (sometime I need to check these out) #btd11
  • If the tools these guys are showing are from their company then I'm disappointed this is turning into a commercial. #btd11
  • Cool, @RemcoOostelaar & @johnbertens are showing tools that worked for them, not a commercial. #btd11 :->
  • Glad to see @lisacrispin wondering about the value of the cloud in #softwaretesting ... It's not scary Lisa ! ;-) #btd11


Another 'must see' (for me) after the Cloudy business. Peter Morgan's "Poor defects BUG me". Peter is very punctual, well organized and well prepared (knowing him from the EuroSTAR 2010 committee). And it shows in his presentations. Everybody got hand-outs of the slides and Peter has really a good fluent story to tell.
Tweets from this track:

  • Now at Peter Morgan's track at #btd11 "poor defects BUG me!" http://yfrog.com/h2mznjj (bit hiding behind the desk) :-)
  • Quite interesting and hilarious components in Peter Morgan's track.. Brilliant! #btd11
  • Peter Morgan: "A large project will stand or fall on its defect tracking mechanism" #BTD11
  • Peter Morgan: "Writing better defects gets them fixed" #btd11
  • "raise problematic items as soon as possible" - Peter Morgan #btd11
  • Peter Morgan on issues, bugs, defects, ... whateva. Great presentation #btd11
  • "track your own defects. Hassle developers, and horse-trade. Buy them lunch" - Peter Morgan #btd11

There are many lessons in this track, if you ever get a chance to see it, please visit it. It's about the impact of good bug reporting, about the agreement of terms for defects used and the expectations regarding bugs/ defects in your organization.


Finally the track-choosing was over. I'm actually no fan of choosing tracks, because although I mostly choose a great track, I can't stop wondering if I'm not missing out on the other parallel tracks (which I'm certain I do). So I'm quite happy that the rest of this day is filled with keynotes.

The first keynote is that of Stuart Reid "Innovations in Software Testing". The talks from Stuart are always under heavy debate. It occurred to me that - even if his talk isn't about certification or ISO - people start (mostly negatively) discussing or pouting regarding this subject in relation to Stuart. (what's up with that?), there are always people who are in a particular mode and are not willing (or able) to break free from this thought line. I even caught a (rather popular) blogger on saying 'I'm not agreeing to this' on a comment which basically implies he's not agreeing on stuff he usually advocates. Stating that testing shouldn't be a profession... well I find this weird. Profession is - for me- most about craftsmanship and for somebody who is a very passionate about making testing all about craftsmanship stating that testing shouldn't be a true profession... hum...well... odd...
Stuart's talk was about innovation in testing, a view into the past, the present and the future. He showed some time lines which in basic showed that some inventions or initiatives that occur at some time, couldn't have occurred when something else hadn't been there to facilitate that development. It was a basic thought, not a whole extensive lecture on each development (like some people pointed out missing the Agile developments). He also pointed out that the testing community is build up in three different levels; testers that shouldn't be called testers, the larger middle bunch and the upper level (people who're into profession c.q. craftsmanship. He also stated (later in the lightning talks) that having a degree doesn't make you a tester, 20% of the pole dancers have a degree (not a pole dancer's degree mind!), 20% of the testers also have a degree (not a testing degree mind!), but that doesn't have to say anything about the skills of that tester (could be biology degree for that part).

Tweets related to Stuart's keynote:

  • Getting ready for Stuart Reid's keynote at #BTD11
    http://yfrog.com/h4k38kjj starting now: the keynote of Stuart Reid about innovation! #btd11
  • Sony lets everyone use blu-ray technology - comparison w/ VHS format success over Beta back in the day - learning fm failure #btd11
  • Watching dyson vacuum cleaner test video in Stuart Reid's talk #btd11 "huge amt of testing... exciting and fun", James Dyson quote
  • Own thought: what if somebody patented 'making fire' or 'using the wheel' , what the world would look like now? #btd11
  • I have not failed, not once. I've discovered a thousand of ways that don't work. Thomas Edison qoute by Stuart Reid... #BTD11
  • innovation comes from failure , so true, this is why companies need to tolerate failure, encourage learning/experimenting #btd11
  • @lisacrispin Innovation comes from learning from failure, not necessarily failure in itself. #btd11
  • @mgaertne @lisacrispin true, but the failure has to occur to learn from it in the first place right? #btd11
  • Celebrate failure RT @lisacrispin innovation comes from failure; companies need to tolerate failure, encourage learning/experimenting #btd11
  • Sure, requires learning from it! RT @mgaertne: Innovation comes from learning from failure, not necessarily failure in itself. #btd11
  • If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. #btd11
  • Stuart forgot to mention that #testing everything is not possible in first place. #BTD11 #BTD
  • http://en.m.wikipedia.org/wiki/Pareto_distribution #btd11
  • Stuart citing Weinberg's Law. Did I miss this one? #BTD11 #BTD
  • @lisacrispin don't forget total quantity management! :) #BTD11 #BTD
  • program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence. #BTD11
  • Now it gets interesting: Risk-based testing leads to context-driven testing. #BTD11 #BTD
  • "High capability employees drive innovation" says Stuart. #BTD11 #BTD
  • Like Stuart's ideas on innovation/diversity but why did his history of testing completely ignore agile? Maybe it's still to come. #btd11
  • @belgiumtesting #btd11 Crosby's maturity - from Stuart - comes back in our talk tomorrow... http://tinyurl.com/5s6g34q
  • #btd11 Crosby's maturity - from Stuart - comes back in our talk tomorrow... http://tinyurl.com/5s6g34q
  • "We need to discuss ideas - we look for like-minded people" I call this confirmation bias. Sounds familiar? #BTD11 #BTD
  • Stuart Reid now talking on CSR? or am I mistaken? #btd11
  • @mgaertne I hope he is telling us to look for diverse viewpoints for discussion? #btd11
  • @lisacrispin Of course he is :) But why does he need to tell me this? I got other interests as well. Seems natural to me. #BTD11 #BTD
  • If you have two people who think the same, fire one of them. What do you need duplication for? Jerry Krause #btd11
  • #btd11 : Groupthinking: beware of enclosure within that group. Innovators also need to look outward!
  • I like the way Stuart gives his presentation. He does not run. #BTD11
  • ok, looks like agile will not be part of Stuart's history of testing. Oh well.
  • #btd11 Good stuff on innovation, creativity in his preso.
  • @lisacrispin Where has that interesting stuff been? Probably before I arrived. #BTD11 #BTD
  • @mgaertne that's to me an obvious difference between vision/thought tracks and informational talks..#btd11 #expectancymanagement
  • it's interesting to look at history of testing, but I'd rather talk about how to integrate testing more into s/w development. #btd11
  • @FunTESTic groupthinking and tunnelvision read "sway the irresistitble pull of irrational behavior" of brafman & brafman #btd11



After Stuart's keynote a very exiting keynote was announced: "Lighting Talks". In this keynote the 6 tutorial speakers (Johanna Rothman, Lisa Crispin, Julian Harty, Stuart Reid, Lloyd Roden and Hans Schaefer) and Dorothy Graham got to have a maximum of seven minutes to talk about their view on 'Looking into the Future'.
I will only post the tweets from the lightning talks, the content can be read at Markus Gärtner's blog (see last tweet)

  • Dot Graham lightning talk: the future of test automation. SHE is talking about agile! #btd11 Read @mgaertne's blog in a little while!
  • Dot Graham: Exploratory test automation! (future) #BTD11 lightning talk
  • There was a big line at the loo! RT @mgaertne: Why are all those folks leaving the lightning talks? I looked forward to this! :) #BTD11 #BTD
  • test automation is free, but only if you're willing to pay for it! Dot Graham,
  • #btd11 ROI on nothing = nothing!
  • Now #btd11 Lightning Talks, Dorothy Graham. Testautomation is free, but only if you're willing to pay for it :-)
  • Future of test automation: significant growth; good news for your manager: good ROI, but investment needed. By Dot Graham #btd11
  • Hans Schaeffer - more ppl call themselves "tester" or something with "test" in the name than 10 yrs ago, good thing. #btd11
  • Hans Schaefer teaches on software testing, but for all the universities he is teaching, only one has testing as a mandatory course #btd11
  • Nobody's timing these lightning talks, that is dangerous. Someone stop me at 7 minutes pls! #btd11
  • On the future of testing: The Vanguard alliance should be mentioned! #BTD11 #BTD
  • Now #btd11 lightning talks: Hans Schaefer. Keywords: accepted, education, cloud, virtualization, availability of tools, possibilities.
  • Hans Schaeffer promoting jobs for handicapped testers like autists! #btd11
  • @lisacrispin lightning talk about Courage #BTD11
  • @lisacrispin talking on the agile value of courage. #BTD11
  • Business people hired us to develop good quality software @lisacrispin Courage #BTD11
  • Now at #btd11 lightning talks Lisa Crispin: Have courage, http://yfrog.com/hsgoofqj
  • @lisacrispin I'll send you a license for Clockwork if you're on a Mac. http://bit.ly/hOES24 #btd11
  • Whoohoo I made it in 7 minutes! #btd11
  • Now @johannarothman telling us about the big bucks she made in her early programming days! :-> #btd11 She musta been good!
  • Now at #btd11 lightning talks @JohannaRothman: the myth about 100% utilization.
  • .@tottinge I showed off the Agile in a Flash cards & talked abt the Courage 1 in lightning #btd11, here's my slide http://yfrog.com/hskavp
  • You can't do much more than about 6.5 hrs technical work per day. @johannarothman #btd11
  • A 100% utilized road is a parking lot - I might add to @johannarothman's Lightning talk. #BTD11 #BTD
  • Does anyone know Julian Harty's twitter ID? #btd11 He just started his lightning talk on pushing boundaries of test automation
  • aautomating usability tests: http://bit.ly/gHWe9d by Julian Harty #btd11
    open source library for finding layout bugs http://bit.ly/ctL50x from @julianharty #btd11
  • Now at #btd11 lightning talks Julian Harty: pushing the boundaries of test automation including heuristics http://yfrog.com/h063cdij
  • Beware of automation bias. Wish @julianharty had done a bitly or tinyurl on that link, I can't possibly type it. #btd11
  • Still at #btd11 lightning talks: Stuart Reid about Testing Professionals, will they still be there in the future?
  • Tired of ppl using "developer" as synonym for "programmer". Maybe a battle I can't win, but we ALL contribute to developing s/w. #btd11
  • I've personally never had trouble being accepted as a professional even tho I'm a tester. #btd11 & have been 1 a long time!
  • @lisacrispin what do you understand by dev and programmer? waht is the difference? #btd11
  • I certainly have no relevant degree level qualification, tho my BS in Animal Science (beef cattle) & MBA have served me well in s/w. #btd11
  • 25% of pole dancers have a degree. Thats the same for testers with a degree... Stuart Reids lightning talk at #BTD11
  • How would anyone GET a degree in software testing? #btd11
  • You can't get a degree in having a good attitude, willing to learn, willing to do any job to help the team, committed 2 quality #btd11
  • Lloyd Roden: "I'm standing between you and a drink." lol #btd11. We're going to play Weakest Link!
  • And last Lightning Talk at #btd11 Lloyd Roden : test cases... Quality & quantity.. The weakest link :-D yeah! Fun!! http://yfrog.com/gywk6uj
  • @silverSpoon that said, even on my team the programmers call themselves developers. So, I think I should give up the fight. #btd11
  • Love Lloyd Roden's happy face tie. #btd11 Outdoes my donkey jewelry.
  • So nice of the #btd11 organizers to give us lightning talkers Leffe beer + awesome chocolate from Leuven! Thanks! The best of Belgium!
  • As usual, @mgaertne has done a great job summarizing the #btd11 lightning talks! http://bit.ly/faKEYN

After the Lightning Talks there was a 'conference drinks' and there was plenty of drinks and some very good snacks handed out! It was very nice to catch up with people, having conversation and discussion, while waiting for a surprise SHOW.

The show started at 18.30. It was improvisation theater and it was absolutely brilliant. (for the Dutchies: it was like the tv show: the Lama's). I found it particularly clever for three Belgians to do a whole improv in English and very much enriched with testing jargon. I had a very good time and laughed a LOT! It was well done and a perfect closure for day one.

After that I had a short dinner at the hotel bar and went to my room, where I ran through my presentation again and went to bed.



Day two: Early science and Excitement

This day started EARLY with a breakfast session, the buffet was open with breakfast stuff and in the expo a small crowd had gathered (which were able to pull themselves out of bed) to go to the first keynote of the day that started at 07.15 hours.
The surprise keynote was all about microchips and the - rather depressing- impact of those developments on testing.
It was amazing to hear about these supercomputers and their ability to calculate.
Tweets from this keynote:

  • #Btd11 EARLY! But at surprise breakfast session about next gen chips and cleanrooms IMEC
  • #btd11 interesting : ExaScience , exascale computers should be available by 2018. Exascale is 50000000 pc's when counting in FLOPs - wow!
  • #btd11 To run an exasystem you need 1.5 Gigawatt of energy: so you basically need a nuclear plant to power it, this is an issue to solve
  • http://yfrog.com/h76izroj #btd11 technology scaling , getting smaller and faster.
  • http://yfrog.com/h32qoscj #btd11 very, very, VERY small technology, chips smaller than virus and bacteria, known, but still impressive
  • #btd11 testing concurrent programs... Non-deterministic , Heisenberg's uncertainty principle... That should give some challenges in testing
  • #btd11 oh boy, you REALLY don't want cache balancing / coherency on the software side: tester's nightmare if no. of caches grow
  • #Btd11 supercomputers: the faster they get, the more hardware failures will occur: up to once every second
  • #Btd11 : "it's very likely that the wear out time of chips will move into its time of usage ..." [quote from presentation]
  • @FunTESTic It looks as if you're the only one awake there :-) #BTD11
  • #Btd11 imagine that your program will behave differently at every use, because of faster chips: that's a tester's challenge on predictability
  • Breakfast speaker fm ExaScienceLab DID kind of depress me w/ news pgms will behave diff every time they run on many core computers #btd11



After that Julian Harty's keynote started. There were quite more people in there than at the first keynote; guess that 7.15 is just too early for most testers at a conference. Julian's talk was quite controversial: "Alternative Testing: So we have to test like we always have?". Julian has an impressive track record being a tester at Google and now at eBay, sharing knowledge whenever he can and even provoking sometimes by setting statements that are - for some testers - unheard of. This track was a bit of the latter. (and detail that most of you not sitting in the front row would have seen: he's presented on his socks :-)) )
Julian did a great job in confusing the audience, letting them think about the content he was presenting, asking questions and showing examples.

Tweets related to this keynote:

  • @julianharty "most of the things we call testing are not testing and should be stopped" #BTD11
  • Morning keynote: @julianharty is challenging the traditional picture of testers referring to Facebook #BTD11
  • @julianharty Speed trumps everything #BTD11
  • @julianharty mariposa botnet story amazing #BTD11 http://bit.ly/dKbmSJ
  • #btd11 now at keynote of Julian Harty, alternative testing: do we have to test like we always have
  • Maybe I'm still not awake, I'm not relating to the keynote well. Doesn't resonate w/ my experience. #btd11
  • @lisacrispin Unfortunately same here. #BTD11
  • @julianharty testing for productivity vs. testing for quality #BTD11
  • @michaelbolton @julianharty just referenced testing vs. checking, making the case for exploration. #BTD11
  • #btd11 http://yfrog.com/h31yyqxj maybe a bit hard to read, but slide from keynote
  • IMO root cause analysis can be beneficial, but teams can get too caught up in it - just experiment w/ a solution. #btd11
  • @julianharty upgrade and roll-back. must be able to roll back! #BTD11
  • #btd11 read the blog about checking vs testing as mentioned by @julianharty here http://bit.ly/dvccu7
  • @julianharty if don't test, get answers anyway. if test, get answers sooner. effective testing gets answers sooner #BTD11
  • #btd11 and another slide: http://yfrog.com/hs57094191j 'what now'
  • James Bach on Jamie Dobson: http://bit.ly/dENYG5 #btd11

After Julian's keynote it was my turn. I did a track called "Unusual Testing: Lessons learned from being a Casualty Simulation Victim". I was - as usual- very nervous and this time particularly nervous. Before this, I did the Software Testing Ethics Debate, but compared to this track thát was quite easy, the audience itself is responsible for most of the content in the debate, but this track is actually stuff with my own opinion in it, and lessons I think are inspirational for 'us testers' :-). Well - after some technical difficulties- It went rather well and I'm actually very proud of the result!
Tweets from my track (proud that is are so much of them :-) ):

  • Now learning from @FunTESTic - "lessons learned from being a casualty victim" (fortunately, from simulation only) #btd11
  • Interesting - casualty simulation is testing - simulation of emergencies to train/test first responders #btd11 @FunTESTic
  • When 1st aid workers have practiced in a simulation, they aren't shocked by real thing, & can function well. #btd11 @FunTESTic
  • thanks @gojkoadzic for figuring out the lights! #btd11
  • use the testing process learned from software testing in an unfamiliar environment - casualty simulation drill #btd11 @FunTESTic
  • still have to set goals, get info, set up environment/scenario, prepare artificial trauma, determine specific. #btd11 @FunTESTic
  • I like that the casualty simulation also involves creativity, innovation - makeup for simulated wounds, setup for crash simulation #btd11
  • making it as real as possible, just like software testing! You have to stay in your role. #btd11 @FunTESTic
  • If you're in a real emergency situation you say "no play" - we might need this in s/w testing too - "no test!" #btd11 @FunTESTic
  • Simulated casualty victims have different goals than real ones. When we test, we have to keep our goals in mind. #btd11 @FunTESTic
  • @FunTestic know your goals, know your roles, stay in your role #BTD11
  • Evaluate - start with good points, end with what to improve. #btd11 @FunTESTic
  • I must say @FunTESTic is a convincing casualty victim in these photos! #btd11
  • Prepare - set up scenario. Specify - mind 1st-aid procedure. Execute - play the role! Have fun! Evaluate - good & faults #btd11 @FunTESTic
  • @FunTestic Casualty simulations actually have some fun #BTD11
  • They actually had a NOPLAY situation in ferry disaster simulation, casualty simulation victims got real hypothermia! @btd11 @FunTESTic
  • Hmmm, @FunTESTic gives birth weekly! To help medics learn to deal with childbirth emergencies. #btd11 ex. eclampsia
  • Video of @FunTESTic's simulated seizure is darned convincing! Now she's having a cardiac arrest. #btd11 She should win an Oscar!
  • You can use a tool - a doll - but a real live person provides better feedback. #btd11 @FunTESTic Feedback is important!
  • The more the scenario is prepared, the better the execution and results from a drill. #btd11 @FunTESTic The victim learns also!
  • Checklisting is a good tool during all phases of a drill #btd11 @FunTESTic (I'm a big fan of checklists myself - read Checklist Manifesto)
  • Better simulation results in better performance and results. Practice! learn the domain! #btd11 @FunTESTic had to learn how to "give birth"
  • @FunTestic Observers help with simulations. yup! #BTD11
  • Communication is a pitfall in every large drill. #btd11 @FunTESTic Use of observers is educational, helps prevent errors (same in s/w test)
  • Use of checklists is crucial in life-threatening situations. It's easy to forget under high stress. #btd11 @FunTESTic Airline crews use 2
  • Interesting - they do drills to test protocols for incoming foreign help in disasters. #btd11 @FunTESTic
  • "we learned much more from the few things that are wrong than the many that went without a glitch" from FLOODEX simulation #btd11
  • The experimentation done with the simulations is fascinating - eg. all-foreign modules, or teams integrated w/ multiple nationalities #btd11
  • Learning follows from mistakes, so we focus on mistakes even though more went right. #btd11 from video shown by @FunTESTic
  • "A first requirement is that participants can see who is doing what and why..." Say what you're going to do. #btd11 @FunTESTic Context!
  • Use of jargons and acroynms where it's assumed everyone understands can lead to miscommunication & disaster #btd11 @FunTESTic
  • @FunTestic what language do you speak? language, terminology, jargon, acronyms. communication is non-trivial. #BTD11
  • If needed, get a translater. Your client may not understand your testing jargon; you may not understand the domain. #btd11 @FunTESTic
  • @lisacrispin mistakes?as a golden medal volleyballcoach once said, always evaluate after a game you won, people are open for feedback #btd11
  • Checklists prevented almost all occurrences of surgery on the wrong eye. #btd11 @FunTESTic (I recommend _Checklist Manifesto_ 4 more ex.)
  • You can have a bad day and miss things, it pays to have someone there using a checklist. #btd11 @FunTESTic not a panacea of course
  • Speak up when you see an issue. @FunTESTic #btd11 (this relates to being a fully engaged team member at all times, never a passenger)
  • @FunTestic Speakup is a powerful tool when you see a problem. (I have a problem *not* speaking up) #BTD11
  • heh RT @johannarothman: @FunTestic Speakup is a powerful tool when you see a problem. (I have a problem *not* speaking up) #BTD11
  • Hobbies can inspire you in your work, and vice versa! Be inspired by things at your job and your daily life. #btd11 @FunTESTic
  • The more "real life", the better the results. Requires skills, preparation & domain knowledge. #btd11 @FunTESTic (words to live by!)
  • Experiment and look at what works best for your environment. #btd11 @FunTESTic Weigh each time what applies best to ur project.
  • Contact @FunTESTic if you'd like to be a casualty simulation victim in Europe! #btd11 She says it's fun... sounds a bit scary to me! :->
  • YES RT @lisacrispin Hobbies can inspire you in your work, and vice versa! Be inspired by things [in] your daily life. #btd11 @FunTESTic
  • @FunTESTic I learned a lot from your talk, interesting new perspectives! Thank you! #btd11
  • #btd11 darn.. I was so caught up in conversation I forgot my chocolates in THE room, save them please! I'll be right back...


Well.. I was planning for an extended break after my performance (there was a coffee break after that). I always need to relax big time after the stress (and I DO get stressed). But directly after my track was one called 'The A(utomation Team' (A-team) from Björn Boisschot and since I'm a fan of 'eighties tv shows' I really wanted to visit this track with a analogy to the A-team. It started with the intro theme of the tv-show and after that the roles and set up of an automation team were shown by relating them to the cast of the A team and their set up of approach.
There weren't actually many tweets of this track, actually two: one of me attending and one about the right tools for the job showing a picture of the ATeam van and a couple of varieties of them (among them a SMART in A-Team style: so cool!)
It was partly my fault, still catching up on the tweets from during my session, listening to the track AND my fellow-tweets were in other tracks.
It WAS however a very entertaining track AND It was - for me not being very actively involved in test automation in my current project- useful to see what particular roles there are, the ATeam analogy made it very understandable for me.




After the - again excellent- lunch, where I was again tempted to pass the healthies and go for the desserts only, I went to a keynote called "Bpost v2.011 - the journey of a complete makeover of a state-owned monopolist" by Koen van Gerven. THis was quite a stranger in our midst. Mieke Gevers explained that it would be a good addition to see info from the clients perspective, what this track was all about. It was quite interesting to see the huge changes at bpost.
Some tweets:

  • Now at keynote #btd11 koen v gerven about BPost v2.011 after good lunch http://yfrog.com/h3nx5nnj
  • I wonder... Is the bpost also using and encouraging twitter? #btd11
  • :-) u don't need a government, but a postal service to be civilized..bpost .. Keynote #btd11
  • Few years ago de bpost was still operating without an IT enabled front office. Complete paper trail. Hard to imagine nowadays. #btd11


Well, I thought it was time for my extended break after this. I was getting tired and found that I was less involved in the keynote I just visited. I decided to take a long break and relax. So I could be me more attentive in the last track and keynotes.

So my next track was one at 15.30: Graham Thomas and "How to suspend testing and still succeed - a true story". Just like Susan Windsor's tracks, I just HAVE to visit Graham Thomas' talks. I find his sense of humor absolutely brilliant, he just has this way of presenting that really appeals to me: it's always fun, always has learnings and well.. .is simple & plain: great to visit!
This track was about suspension of testing, not an easy task when on a project that has a burning rate for resources which would make me a millionaire in less than one months time. It was filled with info on spotting problems and how to coop with them, I thought it was really useful, especially because I'm involved in situations like his project too.

Tweets from the track:

  • http://yfrog.com/h0x2mcgyj Graham, kicking off his track #btd11. Oh, the humor is absolutely brilliant! (as ever)
  • Objectives were reduce lead time, incr. customer satisfaction, highly predictable product quality #btd11 testing as integral part of dev
  • ..the first phase of external scheme testing commenced.."just the right moment for a Testmanager to get involved" [sarcasm] #btd11
  • Really enjoying Graham Thomas his presentation. Love the English humor and the self reflection. #btd11
  • Now watched a movie and moving on to suspension and the definitions of suspension #btd11 (IEEE829)
  • @johnbertens sometimes looks like the twilight zone..;-) #btd11



After the talk from Graham, it was already time for the last keynote of the conference. Lisa Crispin with "Learning for Testers". I 'knew' Lisa from twitter and from the lighting keynote yesterday, but that was about it, so I was very curious what the keynote would be all about.
Well, Lisa is very interactive with the audience, she even goes INTO the audience to participate more. She also had a couple of exercises during the keynote so we were kept very busy during the talk and I was - besides tweeting- very busy writing down notes. One of the highlights from the talk was that we had to write down something we wanted to learn with our e-mail address on it, than we had to exchange that card with a neighbor in the audience. I had a Selenium question for my neighbor and he had a question about Specification by Example for me. (I have been already provided with a solution and in return I bought the eBook on Specification by Example to share with him, learning also by finding the info).

Tweets from the keynote:

  • @lisacrispin keeps referring to @FunTESTic 's casualty victim simulation presentation. Good reason to check it out when you can! #btd11
  • Especially the Daniel Pink talks RT @FunTESTic: #btd11 @lisacrispin tip: learn watching the tedex video's
  • #btd11 http://yfrog.com/h2cvmsgj sheet from keynote by @lisacrispin
  • Learn sign languages to stop the shouting. Nice contribution by an attendee here in @lisacrispin's keynote. #BTD11
  • @lisacrispin innovations arise from a more diverse team #BTD11
  • #btd11 another sheet from @lisacrispin http://yfrog.com/h3vjnrhj
  • A community of practice in your organization is a great way to learn something new. @lisacrispin #BTD11 Agree. Try testingdojo.org
  • #btd11 something I picked up at @lisacrispin 's keynote "be aware of impediments you might have"
  • @lisacrispin create testing communities of practice in the organization. me: aha! just figured something out. #BTD11
  • #btd11 a lot of stuff from the whole conference is now referenced at @lisacrispin 's keynote, it all comes Together :-) really great!
  • #TestingDojos, #WeekendTesting, #WeeknightTesting, .... lots of ideas to learn more about #testing. @lisacrispin #BTD11
  • So many ideas on how to learn, very inspiring! #btd11 @lisacrispin
  • Lots of tweeting buzz from #btd11. Wish I was there to participate.
  • http://yfrog.com/gyqjbej useful resources from @lisacrispin keynote http://yfrog.com/gyuyyhij #btd11
  • What to Offer in Addition to Salary, http://ping.fm/fUKER #BTD11


So after Lisa Crispin, it was suddenly all over. The closing was done by José and Mieke, and some prizes were handed over to people who had participated in a quiz. When leaving the keynote room, all was already in the phase of cleaning up.
I went to the station to catch the train to Brussels Midi to get on the Thalys to Amsterdam.

In the train back I reviewed all the stuff from the conference in my mind. It had been a very fun conference, meeting people, having conversations. The venue was good (especially I liked the fact that I could sprint out of my room 5 mins before the start of the conference :-) ) and on a easy accessible location, food was good and my mind packed with 'thought goodies'. I loved the 'relax' bit of the theater. It's been a good conference in my experience when I'm happily tired out, and I was! (still am!)
When I get the chance I'll certainly go to next year's edition too and I would recommend it to others too. It's a conference packed with high quality tracks and you can see that care has been given to the compilation of the program. Pricing is a bit steep though, but I found this year's program to be value-for-that-money, especially because it's a full-day experience, you don't have to find something to do (besides that you don't have the energy for that :-) ) in the evening except going to dinner (and the hotel bar provides an excellent choice of foods).

So that's my experience of the Belgium Testing Days. It's been a pleasure all and see you at the next conference - signing off :-)

donderdag 10 februari 2011

ISTQB Expert Level ITP- Shout 'nd Pout

Yesterday night I went to the introduction of the brand new ISTQB Expert Level ITP (Improving the Test Process), which is the next level seen from the ISTQB Advanced Test Manager module and a 'tip of the veil' of the ISTQB Expert Level TestManager (also a next step from Advanced Test Manager). It was hosted by TestNet and presented by BNTQB. The setup was pleasant as usual and the topic interesting to say at least...

I was very exited, because rumor had it that this level would be much more about expertise and less about theory. I expected this level to be of such high level that it would challenge me, that I - still relatively short in software testing (from half 2004) - would have to wait until I could apply for such a certificate until I got enough 'years of practice'. I expected it to be some kind of 'Guild'like certificate, where you have a Master-Apprentice style evaluation (with an external/ objective evaluation added of course). I expected it to be of thát calibre that getting certified would really make me feel "WOW, I made it!" and that being certified would really MEAN something. Well... my expectancy management was a bit flawed here...

The first question that popped up in my mind was: "Why a certification on improvement of a test process and not certification on 'test process' itself?". I have been thinking this a lot lately; not only in testing, but in other areas of expertise as well, there is a lot about 'improving', but the set up of a decent/good/etc. process (or product) itself seems to be forgotten. Or maybe current set ups are thát bad that we can only think about 'improvement'.
I set aside that very persistent thought and looked at the slides and listened to the story.

The content itself is something that seemed well thought off. The presentation was high-level here and showed the differences with the content in the Advanced level. Nothing very controversial here (for me at least).

Then it came to the examination criteria. On the slide it stated: a three hour exam, consisting of 1 hour multiple choice and 2 hours of essay. Uh... WHAT? Thát's the exam? I couldn't believe what I was reading.
The next slide got into more interesting part, but that made me very uncomfortable. It was about the validity of the certification. The expert level only lasts for 5 years (that part I find Ok); BUT..in that five years you have to collect 'credits'. There are many areas on which you can earn credits. But some of them worried me, for example: One stated that you could earn credits by giving a presentation on a conference. My first concern here is that currently people are presenting on conferences because they have something to share out of passion, to educate or to really contribute to the community (or when it's a vendor to sell/ demo their product). When you reward people with 'certification credits' for giving a presentation, how many will only submit a topic just to earn those credits? There will certainly will be people who can write an abstract that is good enough to pass all the reviewing in advance and get 'on the program', but then... well I think you can figure it out.

So when the presentation was over, I was left very disappointed. I couldn't believe that my expectancy was THAT far apart from what I just heard. I couldn't believe that all those people who worked on this syllabus would really think of this for an expert certification. I did the ISEB Practitioner (old skool) exam, the one where you had to essay for three hours, not the one with the multiple choice stuff, and what I saw now wasn't that much different and this is the Expert (!!) level.

Hmmm. Okay. So here I am. Discontent about what I heard. Writing all this in a blog, still very disappointed. Is it justified? Why am I so far off in my expectancy and what I heard? Maybe it's because of what is told or what is NOT told?
The syllabus is download-able (http://istqb.org/display/ISTQB/Expert) . So let's take a look at that...

One of the paragraphs states:
"In addition to passing the exam, proof needs to be provided of practical working experience in the testing field in general and specifically in the field represented by the Expert Level Module before the Expert Level certificate is awarded. In addition to passing the exam the following requirements apply:
- at least five years of practical testing experience (CV needs to be submitted including two references)
- at least two years of experience in the Expert Level module (CV needs to be submitted including two references)
- at least one paper written and published, OR a presentation is given at a testing conference covering an Expert Level module topic. "

This wasn't mentioned in the presentation. The emphasis was on the content and the 'credits' to be earned in the five years after passing the exam. The most people I spoke where very much interested however on the examination criteria themselves (and HOW it was examined) and they were covered but it didn't state (or at least I didn't picked it up) these additional preconditions to be able pass the exam.
This makes it at least more valuable, for me.

If you have a threshold set of a minimum level of expertise before even be able to do the exam, that would certainly make it more Expert Level to me. The thing is however still how to evaluate the entry-criteria set. So I think there must be objective points to evaluate these criteria.
Example: If I have 5 years of experience in testing and 2 years in the Expert Level module topic that still doesn't say anything about whether I'm any good. I could have been sitting on my ass all the time. The references are making it more solid, but they could be friends that humor me. The paper/presentation can be done too; but maybe people ran out the room because it was too terrible or too rubbish to listen to. So I still meet the criteria (pur sec) but I'm not much of expert material.
So I plea for a sturdy and strict policy to evaluate these entry criteria and have minimum requirements to have an Expert Level candidate to be able to participate in the exam. It takes more work I agree, but at least it makes the Expert Level a real solid certification!

Well at this time only the syllabus is published. The ISTQB is still figuring out on the course itself and the examination. Until it's there we can shout and pout. But we can also help the ISTQB in making the examination a real valid one that means something, I think this will only be appreciated by the people who have worked very hard on this material, and I think this certification should be one that makes the certified Expert an Expert indeed.
(and I'm not saying that NOT having a certification means your NOT an expert :-) )

For now, I'll 'dive into' the syllabus and see what's really in there and maybe find what I didn't hear yesterday, have more understanding of what's it all about, before I 'shout&pout' :-)