Agile Technical Documentation

By Jean-Luc Mazet

Bookmark and Share


Click a link below to jump to a particular section; click any "CONTENTS" image following a section heading to jump back here.

Introduction    Back to Contents

Agile methodologies have been focused on and mostly applied to software development or product engineering. Agile was intended to encompass all areas associated with software development. It is not until recently that some of the practices have truly extended to other areas which are part of product development, such as technical documentation.

Content has become more malleable and requires more touch points than in the past. Agile methodology allows for rapid redirection of projects and resources to address this change. Technical communicators who work with Subject Matter Experts (SMEs) need effective communication methods and collaboration spaces to develop their content. Tasks-based writing and the importance of low-touch, high-return collaboration for content creation and review is a key factor to applying Agile practices to documentation projects.

Technical communicators can use Agile practices, such as Scrums and user stories to create a progressive canvas on which they develop customer-facing documentation, like user manuals or online help. This presentation will detail and explain the approach and the mechanics to successfully integrate Agile documentation to Agile software development projects.

Agile and Software Development    Back to Contents

It is not the purpose of this paper to go into too much detail about Agile methodology in software development. You can read about the history and terminology on the Agile Alliance Web siteExternal link or on online encyclopedia sites. I am simply going to introduce the methodology and some of its terminology that is useful to understand this presentation.

In the mid-1990's this methodology emerged as an alternative to massive development methodologies that favored processes, tools, and rigid planning. In the new century, several members of the new methodology community met and formed the Agile Alliance and later created the Manifesto for Agile Software DevelopmentExternal link.

The principles center on incremental, rapid software development with a vision that delivers value; a potentially "shippable" product or "demonstratable" functionality. This value can be demonstrated at the end of each iteration and receives feedback and, if needed, course correction from the stakeholders. The framework relies on an iterative development process, close daily cooperation and communication, adaptation to change, trust, and simple efficient design.

Some useful terms to be familiar with are:

  • Pig: Person committed to building the product on a regular basis.
  • Chicken: Person with an interest in the product and what the pig delivers.
  • Product Owner: Representative of the customer's interest in the product. This person needs to be available at all times to answer questions and make decisions. Sometimes called the "single-wringable neck."
  • Stakeholder: Internal or external customers and vendors or suppliers who will not be directly involved with the project, but may weigh in on the results of each Sprint Review.
  • ScrumMaster (or Facilitator): Daily Scrum Meeting facilitator. Person who removes impediments (distractions, external influences) and enforces Scrum practices and rules.
  • User Stories: Customer-centric, product requirements or features that can be summarized into one or two sentences. For example: "As a user of this product, I want to be able to log in and out from a central point."
  • Sprint: Work iteration that lasts between 14 to 30 days.
  • Sprint Planning Meeting: First meeting at the beginning of a Sprint to negotiate the work that can be achieved during that Sprint.
  • Product Backlog or Backlog: Prioritized product requirements for customer-deliverable features and internal technical requirements.
  • Sprint Backlog: Represents the tasks for the Sprint and the Product Backlog items.
  • Daily Scrum Meeting or Standup Meeting: the short (15 minutes) team meeting where each "pig" participant reports their progress from the day before, what they have scheduled for the day ahead, and the impediments that prevent their progress. "Chickens" listen in and do not report status.
  • Sprint Review Meeting: Held at the end of the Sprint to show the accomplishments of the team, usually a prototype or potentially shippable product that demonstrates the tasks and goals set at the beginning of the Sprint.
  • Sprint Retrospective Meeting: Held by the team and the ScrumMaster after the Sprint Review Meeting to assess what went well and what can be improved for the next Sprint.

This is just a small sample of the terms used. To learn more Agile terms, refer to the Scrum Alliance Web siteExternal link.

In comparison with other methodologies, like Big Design Up-Front (BDUF), Waterfall, or Plan-Driven, Agile is more adaptive. It allows for redirections of the project and product to make sure it adapts to changes in the development infrastructure, organization, and other areas that may be impacted directly by customers or changing requirements.

Agile and Technical Documentation    Back to Contents

After working with developers in an R&D environment, it became apparent that this methodology could be applied to content development as well.

Make sure that you have all your allies on board. This will serve as the foundation to your successful Agile technical documentation framework. You need to convince your management and your team (content developers and editors) to use this methodology by pointing out the benefits and successful outcomes of such an approach. Once you have buy-in, it is easier to move forward and get the team used to the Agile framework and working principles. The key in this effort is to avoid "selling" it. If needed, it is acceptable to go into "stealth mode" and not overuse the Agile vocabulary. Make it part of a pilot or non-critical project at first. Then, you can show the proof to the skeptics by highlighting the results and benefits.

You can talk to your management team in its own language as well. If someone wants a GANNTT chart, some of the tools allow you to export the data and provide that type of chart. If someone wants to have more detailed information, show them the benefits of getting regular progress. Get them interested in the results and how much they want to get involved in the Sprints. You can also spark interest by asking them whether they would like to come to the daily meetings.

To get low-touch, high-return in your content quality and accuracy, you and your team need to rely on the 5 values of Scrum: openness, courage, respect, focus, and commitment. They help drive the behaviors in your group. Give your team members what they need and trust them with the delivery and getting the job done. Let them self-organize. Note that this requires senior-level personnel and mentalities. The best designs and requirements come out of self-organizing teams; they have no limits and know what they can do. It is easier if you know what you need to deliver and get there on your own rather than being guided every step of the way. Bring in your team members early in the project, since you want them to be well integrated and part of early planning meetings.

The content development mechanism lends itself to the same Agile principles as software development. In traditional documentation models, those I like to call "hybrid" models (relying on proven DTP tools and infrastructures), the team either creates new or updates existing content and can participate in the Sprint activities alongside other functional groups. They can take part in user stories and come up with documentation for each story linked to a specific feature or functionality. Details on how this is achieved are listed later on in the presentation.

In structured, semantic documentation models (usually driven by reuse, repurposing, and multi-channel publishing with XML and associated technologies), the team relies on chunking, inclusion mechanisms, information management tools, and topic-centric writing to deliver the content that is appropriate for each user story and functionality in the product. This model works well with Agile methodology due to the topic-based style of writing that can be used. Each piece of information is authored to support a help system (the building block of your content and what your users refer to first). This content can be leveraged and reused inside of a user's guide with additional conceptual information to complement existing help system content.

Where you might encounter some difficulties and have to be creative in the way that you work and develop your deliverables is when you have distributed teams. Agile relies on small, co-located teams and you may work in a global market and in different time zones. What works well, especially if you are part of several, geographically distributed projects with larger teams, is the Scrum of Scrums, an overall Scrum meeting, where you send representatives of your functional area to report on your progress. Separate, local daily Scrum meetings can still take place; the Scrum of Scrums coordinates the efforts and ensures that nothing is left unaddressed.

Reviews are part of another challenge in Agile technical documentation. Editing passes will have to be on-going, especially if you have a technical editor on staff. They become Sprint-based, and the overhead may become too much for the editor. Peer reviews are an alternative to formal editing, especially with senior content developers as part of your team. This reduces the impact on the formal editing without compromising quality. The final document review can happen during the "hardening sprint" (a.k.a. "commercialization sprint" or "product release sprint") to ensure the consistency and flow of the assembled documents. This is the last bottleneck before the gold build and CD or DVD release, the bill of materials, and documentation printing and publishing.

When you add localization to the equation, you should be aware of a few more risks, especially if you want to follow Agile methodology and localize during each Sprint. This approach can result in sky-rocketing localization costs due to incessant changes in the product and the associated impact on the code and its documentation. You need to minimize the amount of changes that would result from Sprint redirections and product changes in early Sprints. To mitigate the risks, you can create an "epic" or "overarching" user story for localization. This user story starts when the code and documentation are stable enough to be localized. The product is still internationalized, as part of each user story; ensuring that the code and documentation are localizable.

Getting Started with Agile Technical Documentation    Back to Contents

As we have seen in the previous section, one of the first steps is to get approval and buy-in from all the participants or interested parties. Once you have reached that consensus, you can start working on the mindset and framework.

Once you have reached an agreement, you can consider these pointers and best practices:

  • The environment or infrastructure and tools that you are using do not dictate the Agile approach. Some environments will be more conducive to Agile authoring, such as structured, semantic models. You should not discard the methodology because you are not using such environments. The key is to use a task-based authoring approach.
  • You also need to make sure that the feature is ready to be documented. From that starting point, you can create or update your content based on user stories and validate against product features.
  • Talk to your co-authors or team members on the project. Set up a collaborative and even pair authoring environment. Report your progress and discuss roadblocks.
  • If you would like to test your work, at first reading each others' chapters maybe the way to ensure that you have content ready for the user story that you are documenting. Then, when your document reaches a more mature state, you can test output conversions or assemble the table of contents and index.
  • Later on, when you have more content, you can convert it to online help and see how it integrates with the product. This will be the true shippable product of your documentation efforts. At the Sprint Review meeting, you will be able to show other functional teams - and ideally the customer - how you resolved an issue (an "impediment") with a new technique and bring out the "cool factor" in your work.
  • Once your team is self-organized, it can concentrate on the documentation that will provide the most ROI for the stakeholder and ultimately the customers. Your team is the best suited to define the deliverables, the audience, and when and where technical reference versus online help is more effective.

There are a few pitfalls that you should also be aware of:

  • You should understand that your documentation, just like the code, can be discarded at any point during the project, should the product owner decide that the project needs to go in another direction.
  • Starting writing right away may prove to be too difficult due to the nature of the first few Sprints and the instability of the product (if it is a new product). Try to go back to more traditional authoring techniques, like creating a high-level content structure or content "buckets."
  • In the first few Sprints, development infrastructure tasks may lead to a slow down in features that can be documented. Concentrate on your own infrastructure and create a robust, yet simple information architecture. Focus on making your documentation modular and ready for reuse and repurposing.
  • When your Sprint velocity increases, you may be tempted to create large amounts of documentation. Instead, start small; do not write an administrator's guide with all of the conceptual information right away. Focus on the smallest unit of content, for example an online help topic. Then, expand that to include more explanations to create your printed administrator's guide or even to include some of the online help steps in your troubleshooting guide.
  • Do not get pressured into delivering large amounts of content, because your SMEs may want more documentation. There is a tendency to give them what they want. Refrain from doing that. Instead, tell them that you are following development and test, and that you are only developing content units suitable to use in correlation with the user stories. Create an "epic" user story to take care of the overall documentation set and to make sure that all the pieces fit together after assembly.

Working with Agile in a Single Source Environment    Back to Contents

We have established the notion of two content authoring and publishing environments: "hybrid" and "structured, semantic." The hybrid model consists of traditional desktop publishing tools and infrastructure. The structured, semantic model relies on XML and related technologies. Both of these models are set up for single-sourcing and reuse, maybe to a different degree, but nonetheless they can support the Agile methodology for technical documentation purposes.

In hybrid environments, the way that we traditionally author is broken down by deliverable, manual, chapter, and it may silo content developers. The mindset needs to change; you are part of a small team and you develop a piece of the user assistance, not documentation for a specific user's or administrator's guide. Your piece is part of a larger puzzle that will take shape as the Sprints progress. Even though you may not be able to use content chunks, like in XML, you still need to look at sections and procedures as the main content units that are required to complete the user story task at hand. To help with this shift, deliver small PDF files or HTML output of your sections and chapters for review.

In XML environments, the approach is different, since content units are already smaller and the assembly mechanism makes it easier to pick and choose from those units. For example, in a DocBook modelExternal link, you can use XIncludes or entities to assemble your content or pull a section from a team member's content. This will test whether or not you know your content well and whether or not you chose the right information architecture. When using the DITA frameworkExternal link, the mindset becomes easier to impart, since the model already incorporates information typing, topic-based authoring, and content assembly. You can divide tasks and also share your content to get each feature documented separately, yet collaboratively.

Content types in both models play an important role in deciding which approach to take. The approach to existing and to new content will differ slightly. With legacy or existing documentation, you may have to review each deliverable and look for the information that will either be updated for the Sprint or that may need to be added to supplement what was already written for the previous release. Depending on the model, you can use conditional text, conditional processing, or content profiling to provide only the information that the SMEs need for that Sprint. When starting from scratch and developing documentation for a new application, the slate is blank. You can build everything from the ground up, following the principles and best practices explained in previous sections.

With both models come a myriad of tools, such as content management systems, content editors, scripts, and transforms. Although these tools are crucial to the mechanics of content development and publishing, there are also techniques that your team needs to learn to maximize the efficiency of small, cross-functional teams. One of them involves working closer with human factor engineers, developers, trainers, and quality assurance team members. The daily Scrum attendance and information sharing will be conducive to frequent document reviews and progress sharing that will lead to faster information flow. A content developer may also be asked to work with a human factor engineer to provide input on a specific area of the user interface or phrasing for tooltips. Knowing about impediments at the daily Scrum is a good way to get things resolved fast, even if you have to take them "off-line" with a developer or another team member. You will notice the cross-functional boundaries dwindle and in some cases disappear as you work closer together.

Agile Documentation Project Highlights and Lessons Learned    Back to Contents

From the Agile projects that I participated in, I have been able to extract a good feel for the key components for each documentation phase in the project.

As pointed out earlier, an important element for a successful Agile technical documentation project is the team selection. Factors that come into play are the familiarity with other functional team members and with Agile authoring concepts. A strong, self-organized team will adapt better to the rapidly changing landscape.

Appropriate user story sizing during Sprint Planning is another important aspect of the technical communicator's job. Give enough "weight" to your Sprint deliverables and do not let yourself be influenced by another group's sizing. Make sure that all of the infrastructure tasks, such as setting up the authoring environment, acquiring and configuring test servers or other needed equipment are part and parcel of your Sprint tasks and backlog. This will allow you to effectively account for those when time comes to generate the Sprint burn-down chart.

Release planning is another key component of Agile projects. So many teams do not know how many Sprints the project is going to last. This is a reason to stress the early participation of the documentation team in Sprints to be able to fully and accurately assess the amount of total effort that the project is going to take.

I have already mentioned the wealth of information that can be gathered in the Daily Scrum meetings. Stand up meetings, even for distributed teams, are the best way to get the pulse of your co-workers. It is where you can step in and remove impediments as well. At times, you will find out that one of your own team members may have an impediment for which you have a solution. You may also learn about a user interface change or hear of a new test server that you can use to validate your documentation.

As far as content development goes, there are several items that were germane to the success of the overall project and individual Sprints. While creating content, from tooltips to lengthy administrator's guides, we noticed that Wiki-based authoring was the most expedient way to generate user story content without knowing extensive tagging languages or template styles. There is also no output creation, so you skip several steps to convert your source content into PDF or HTML. What a concept: your content becomes the pillar of the Sprint's deliverable, not the presentation or output formats. If you are in-between authoring environments or are researching a new tool, Wiki content allows you to avoid being tied to a specific authoring and publishing model. You have time to carry out your research and evaluate your options.

Wiki gives true meaning to collaboratively authored content: all the functional groups can see the content right away and provide edits and add technical accuracy. In effect, you have expanded your resource pool across all functional teams. Coupled with a good forum, you are ready to take on and document even the most complicated feature in your application. If you are looking for integration with industry-recognized models, such as DocBook, some of the Wikis provide DocBook plug-ins that allow you to author content in DocBook grammar and get on-the-fly rendition in HTML while being part of the Wiki framework and resources. You can also convert your content with scripts that will facilitate content loading into a CMS.

Another time to focus on getting the most out of your interaction with your peers is during the Sprint Review and the Sprint Retrospective. These two meetings provide the essential feedback mechanism and promote team learnings that can be used in the next Sprints. The feedback mechanism and steering which result from these meetings will support the direction for the subsequent Sprints. One of these meetings is for stakeholders and the other one for all the team members involved in delivering the potentially shippable product. Content developers will feel valued and appreciated, as part of a larger team and an integral part of product development. For example, I sometimes demonstrated the progress that we made in our infrastructure and environment or a new transform that we had just completed. We also received praises from developers for the tight integration of our content with the application. The result is that your documentation adjusts faster to application changes than in traditional models. It becomes lean and streamlined, making all documentation efforts count and creating an engaging and motivating working framework for all participants.

Your organization may also notice cost benefits for localization. Your source language documents are better structured and more concise, since content is the focal point of the writing process and task-based authoring only describes features that are released during each Sprint. Your technical documents' organization and accuracy is no longer tied to lengthy reviews at the end of a project when everyone just wants to "get it out the door." As the information that you produce becomes the source of daily discussions, you will soon discover the value of Agile methodology. Another value lies in the fact that when you have no more Sprint tasks, you can sit with a user interface or core server developer and learn more about the front end or back end that you are documenting. You can do the same with QA or testing team members. Ideally, developers could help out with the documentation, technical communicators with some testing, etc. This allows you to expand your product knowledge and skills while keeping your documentation up-to-date as the project evolves.

Conclusion and Key Takeaways    Back to Contents

As a conclusion, see how Agile methodology can serve your technical documentation needs. Adopt it and adapt it to your specific needs. You can also expand Agile to other areas as you complete more technical documentation projects using this methodology. You can bring in training specialists to leverage your content and to help you create training-usable content units within your technical documentation. The same can apply to highly-customized consulting or professional services' documentation deliverables and even to marketing collateral and technical white papers.

In closing, I would like to provide some key takeaways:

  • Convince: Make Agile part of your organization's planning. It is not a "migration" to Agile. Use Waterfall or executive-type language when needed to ease the adoption rate and buy-in.
  • Create visibility: Communicate and be a proponent of Agile without systematically "pushing" the methodology. In a nutshell, make it low-key.
  • Believe in the vision: A robust vision is the root of a viable product or solution. Trust the experts to make the right decisions and back them up with the appropriate amount of research and facts.
  • Use meetings sparingly: Take work out of meetings and back into the actual content development. "Time box" some of your meetings and discussions.
  • Remain fluid: Be prepared to redirect your efforts. Listen to redirections and feedback. Approach challenges with an open mind. Question and learn.
  • Choose your team well: Rely on senior-level members on your team. Promote coaching and junior personnel training during an Agile project. This helps expand your resources for the next project.
  • Support self-organization: Make the team find its strengths and core competencies on its own.
  • Be a minimalist: Focus on the smallest units of content for each user story and expand your documentation with contextual information as needed. You will never have to write around a feature that does not exist.
  • Trim the excess content: Documenting complex systems like enterprise software does not mean that the usual technical documentation will be discarded. It means that you will concentrate on documents that fulfill a specific, immediate purpose, like online help or tooltips. You will know when to expand or supplement these with technical reference or troubleshooting guides.

References    Back to Contents

  1. http://www.agilealliance.orgExternal link
  2. http://www.agilemanifesto.orgExternal link
  3. http://www.scrumalliance.orgExternal link
  4. link
  5. link

Additional References and Readings:

  • Agile Documentation A Pattern Guide to Producing Lightweight Documents for Software Projects, Andreas Rüping, John Wiley & Sons, LTD, 2003.
  • Agile Project Management with Scrum, Ken Schwaeber, Microsoft Press, 2004
  • Agile Software Development with Scrum, Ken Schwaeber and Mike Beedle, Prentice Hall, 2002

After completing his graduate studies (Masters) in linguistics, civilization, and technical interpreting and translating in France, Jean-Luc received a position to teach at the University of Texas at Austin. He later took a position as an IEEE conference manager which allowed him to further explore project management and technical communication. In the past two decades he has managed and contributed to several hardware and software documentation and localization projects for companies of all sizes and in all industry sectors. More recently, as Information Architect and Localization Specialist at HP, he has worked on the team's direction, vision, and strategy, including infrastructure, tools, technology, best practices, and processes for Technical Communication and Localization. In the past three years, he has worked on and contributed to several Agile project teams. He has piloted, developed, and implemented the Agile technical documentation practices detailed in this presentation in real-life projects. He is a Certified ScrumMaster (CSM) and Certified Scrum Practitioner (CSP).

Jean-Luc Mazet, Localization Specialist
Hewlett-Packard Company