Thursday, May 16, 2019

A bit about us...

We seldom speak much about the team behind EJBCA or about PrimeKey Solutions on a whole on this blog, so for those interested I'd like to write a bit about us and our culture for those of you who might be interested.

The Evolution of a Company

PrimeKey Solutions was founded way back when in 2002 by a small group of developers who saw the need for an Open Source CA solution, which was innovatively named EJBCA. While we're great at many things, we do admit that naming products is maybe not one of them - I do bid you remember that this was the tail end of the IT-era; it could have been way worse. 
PrimeKey's logo ca 2005
FOSS was an important concept from the very start - both from a practical viewpoint (don't trust anything that you can't verify), but also ideological. PrimeKey was founded on the principal that cryptography (and by extension PKI) should be widely available, so was from the very start founded on FOSS principles, and PrimeKey have ever sought to make use of open source, as well as contributing to the projects that we use (such as BouncyCastle) both in code and monetarily. 
PrimeKey's logo ca 2014
PrimeKey has since then grown and evolved, from being a tiny one-room-everybody-does-everything-company to what it is today: a steadily growing, +100 person company with offices in three countries, specialized staff and specific roles going forward. 
Our logo since 2017

The Evolution of a Team

I joined PrimeKey back in 2010, back in the heydays of the entire company sharing the same volume of air during a working day. Except for our then CEO and a couple of other staff, the bunch of us (<10 people) mostly filled the same roles to varying degrees: developer, professional services, IT, support, QA, documentation, tech sales - there was little delimitation and even less structure. You did what was needed and learned the skills required to overcome the next task at hand. We were happy in those days, though we were disorganized. 

I was originally hired as a developer, but back in those rousing days the prospect of travel appealed to me as well. Between writing (and refactoring) mounds of code, working with customers took me from Ankara to Antwerpen, from Brussels to Baku. My first major project was working with our upcoming Common Criteria certification back in 2011, which taught me the joys of writing months worth of pointless documentation and speaking to auditors.

What I'm getting at was that we existed in mostly a state of unmitigated pandemonium; working on customer requirements as they arose, our own fancies and interests, putting out wild bush fires and occasionally lighting them. At a tech conference in 2012 I described our work style as "Controlled Chaos", which very much makes me cringe today. 

This finally started changing round about 2016 or so, when we were faced with a few issues:
  • We were evolving as a company, moving more towards corporate customers. A couple of bad experiences of letting customers specify implementations, implementing those specs to the letter and then discovering that the customer had asked for the wrong thing showed that we needed to involve ourselves more in specification work. 
  • While the free-form organization was great for allowing the removal of tech debt on a minor scale, non-critical refactoring that would have required major effort were difficult to organize. 
  • Our roadmap was basically just the JIRA backlog, meaning that release dates were largely hypothetical concepts. 
  • Long release cycles (3-4 months) coupled with a lack of issue slicing and cycle breaks (such as sprints) lead to monolithic commits waiting for months on review, occasionally turning out long after the fact to be unsatisfactory or incomplete. 
We direly needed to change. I won't go into the whole process and evolution of how we got to where we are today (though I might write that as a blog post at a later date), I'd love to describe how the EJBCA development team works at present. 

The Team(s)

The goals of our teams are to be as self organizing as possible, wherein each has their own integrated Scrum Master in order to organize and synchronize the teams around common goals. With QA as integral parts of each team, we've started moving away from testing as a waterfall feature and instead testing proactively (and most importantly, just testing). 

The teams each maintain their own culture and rules internally, and sync one day a week in either a general tech meeting to discuss ongoing tasks, hinderances or questions, and have a common retrospective at the end of each sprint in order to evaluate what went well and what went poorly. 

Our Roadmap and Development Cycle 

My role as Product Owner is primarily to maintain a vision and a roadmap in order to help the teams focus on the most important requirements for the moments, to give guidance and answer implementation questions that the teams can't answer themselves. Thus my main avenue of communication with the teams is maintaining and constantly updating the roadmap, which allows them to on their own prioritize and sort the backlog (with some input from myself). Versions are set for features and fixes only as they enter the near horizon (except for features with fixed dates such as customer implementations and specification implementations), which sends less false signals to customers in regards to completion dates, and gives a truer image of what can be expected in each release.

The teams today follow three week development sprints, at the end of which all issues need to be resolved, reviewed and closed. Issues are sliced down to a three days or less if possible. This cuts down on both issue size and time-to-review, letting us catch design errors earlier, before they propagate. It also eliminates the large degree of release date uncertainty due to the large grey blob of review time (and resulting fixes) always invariably plaguing the end of each release cycle. Besides the retrospective mentioned above, sprints often end with an internal sprint demo to show what's been accomplished, and a release demo post release to show off for the rest of the company what's new. 

The Result

In the last couple of years we've taken great leaps forward in terms in terms of reliability, predictability and above all quality. While we still have a long way to go, we are extremely pleased with the results so far.

The Evolution of an Office 

The Stockholm office has changed addresses four times in nearly 20 years. The gang started off sitting in what amounted to a shoebox, before moving to slightly bigger digs in 2009, which is about the time I came into the picture.

The new office was slightly larger than the old one, but the entire team still shared a single room (with an office for our CEO and VP Sales), happily all breathing the same air for the entire working day. 

My desk back in 2010, with era-appropriate camera filter and still relevant stacks of DVD blanks. 
We shared an office back then with two other companies, with constant bickering over which coffee machine to lease (bad coffee leads to bad code - you can quote me on that). We were on top of a children's osteopath, so on clear days the workday was hellishly audioscaped by the little moppets' wailings up through the ventilation. Good times. 

With time we came to outgrow that office, first shifting out one of our office-mates before getting outshifted ourselves. We'd grown to expand over two rooms, and there was as little piece and quiet as there was space to move around the tightly packed desks. Upon finding new space in 2016, I sat down and wrote a manifesto of office design, arguing that the current and continued practices of open offices was provably based on faulty theories on workspace design, citing the esteemed Joel Spolsky, among other sources. I ranted! I raved! I built barricades out of hutches and obsolete cryptographic digests. Join me! I called. Let us throw off the yoke of office plan oppression, seize the means of production and be allowed to work without external interruptions! 

But one colleague heeded my call. We were given our own room in the new office and told to shut up. We were happy. 
The author exploring his new digs. New feats of concentration were attained in this small room. 
Come this year (2018) we had once more outgrown the previous office. PrimeKey's success in the last few years led to huge growth for the company, and we finally expanded into having proper teams handling support, integration and sales - by extension allowing us to hire more developers and QA staff. Management located a new place only a stone's throw away from the existing one, this time moving us into a complete floor of a proper office building, complete with proper server room in the basement, a view and space for luxuries like a gaming room. 
Our office since 2018
A gaming room with a foosball table, the true sign of prosperity
Upon planning the move our CEO Magnus very graciously (and wisely) asked a selection of employees our thoughts on the new office and ideas for an office plan, so for the developers we suggested a scheme that lies more in line with both our own philosophies and with what is considered modern practice. We actively rejected any suggestions for open office plans (or its blasphemous mutated step child, hot desking), but instead suggested that we put up walls or roof-to-floor-dividers, and space them out to create reasonably sound proof work areas containing 4-6 desks, which we consider the ideal team size. We felt that this would have the following advantages:
  • Humans are social creatures, and do well from interacting with others. Development work (particularly in cross functional teams) requires the ability to hold spontaneous meetings and interactions. For teams to form bonds they need to interact, tell in-jokes, form their own culture. 
  • Development work generally requires large degrees of concentration due to the mental effort of visualizing complex solutions, and unwanted distractions (others' phones, distant conversations, etc) lead to stress, anxiety and generally lower productivity. 
  • As humans we tend to nest in order to create a sense of belonging and familiarity in our surroundings, spreading around us items both useful (that extra phone charger) and decorative. We tend to do so in small groups as well, so this scheme would allow each team to be able to decorate their own spaces as they see fit. 
In addition, the additions of walls and partitions would allow each team to have a dedicated whiteboard to use while collaborating on features, a large TV monitor to statically display information (such as CI machine status, roadmaps or JIRA boards). In the end I have to say, management delivered beautifully:
Team Alice's work area

The home of Team Bob
The dwellings of the SignServer Team
While it may not look like much, these spaces serve us perfectly in allowing each team to concentrate on problems and issues, allowing collaboration and social interaction while letting those needing to bite down on a problem concentrate while avoiding the angst-inducing sight of cubicle farms. 

All in all, were constantly evolving here at PrimeKey - analyzing and reevaluating what makes us better as individuals, as teams and as a company.

We're already doing what we love, and now we're a huge leap forward in how much we love doing it. 

Cheers,
Mike Agrenius Kushner
Product Owner EJBCA