Drupal Community Pilot Survey
In fall 2013, me and Adi Meir, a colleague at the Bentley User Experience Center and classmate at Bentley, collaborated on a survey to study the motivations of Drupal contributors. Specifically, we were looking at a couple of fundamental research questions:
- What reasons do people have for contributing code to the Drupal project?
- What other types of contributions do people make (e.g. design, documentation, running events, etc.) and why?
- Are there people who want to contribute, but don’t? If so, what factors might be preventing them from making contributions?
To research these questions, we first looked at representative studies of open source contributors to find sample questions that could be adapted for our survey. After some deliberation on the nature of the data we wanted to collect, the FLOSS-US study (David, Waterman, & Arora, 2003), was chosen as a starting point. We also asked participants to report on their specific “role” within Drupal, e.g. developer, designer, themer, etc.
Finally, additional questions were asked, in a separate part of the study shown only to those who indicated that they had not contributed, but would like to, to assess potential reasons for not contributing (e.g. “I don’t know how to write modules or code” or “It’s too difficult to collaborate on my type of work in Drupal). As a control, respondents in the non-contributor category were asked to explain, in their own words, what has prevented them from contributing. On a separate screen, they were asked to identify a list of factors related to their decision not to participate, and rate their importance.
The survey was distributed in fall of 2013 via my personal Twitter and Facebook accounts, and forwarded via social media by participants. In total, we received 100 valid responses within a period of approximately 1 week (proof of the awesome generosity that exists in the Drupal community, if you ask me).
As expected, the majority of respondents (68%) considered themselves developers or site builders, while themers, designers, usability professionals and “other” (documentation, community support/outreach, etc.) represented the other 32% (see Figure 1).
Figure 1. Breakdown of survey participants by perceived Drupal role.
In terms of contributions, this sample represented 57 code contributors, and 21 non-code contributors, with most contributors providing both code and non-code contributions. Prior to analysis, contributions were coded into four categories. Code Creation refers to contributions that involve the actual writing of code, i.e. writing modules and patches, or contributing to core development. Code Health refers to providing support for increasing the quality of code, i.e. by writing documentation or reporting bugs. This represented the largest number of contributions (113 in total). Community Health refers to providing support, e.g. mentoring others, volunteering at events, or providing support in the forums and IRC. Lastly, UX/UI refers to design-related contributions, such as UX Design and usability testing.
While the number of contributions related to community and code were roughly similar in our study, the number of UX/UI contributions is relatively small. This result is interesting given the attention that UX and UI has received in the Drupal community. If the number of designers is expanding (as evidenced by the growing number of submissions in the design/UX tracks at Drupalcon, and the growth of attendance at Design 4 Drupal Camp last year), why are there comparatively few contributions of UX and usability work?
Figure 2. Breakdown of types of contributions provided by survey respondents.
Feedback from non-contributors
Aside from looking at the nature of contributions in the Drupal community, an additional goal of the pilot survey was to specifically look at Drupal users who wanted to contribute, but had either not been able to, or had chosen not to. To go a bit deeper into this population, we asked only those respondents who said that they had not "contributed code or other volunteer work” to the Drupal project, and that they had thought about contributing, the following question:
Please say, in your own words, what has held you back from collaborating on Drupal.
The free text entry yielded a total of 19 responses, which varied in length from a single word to a full paragraph of commentary. Among these responses, 5 focused on not having time to contribute (e.g. “Don’t have a lot of free time”), and 8 centered on being put off by either the perceived complexity of the process (e.g. “Not sure how best to do so or where to get started”) or the perceived attitudes of Drupal developers (e.g. “my last impression was that it seems like everyone in the community speaks only in php”). Most importantly, however, (12 of 19) respondents felt that they did not have the skills (i.e. coding skills) required to contribute to Drupal, and/or that skills unrelated to code were not valued by the community. Some examples:
"Don't feel that my skill set is advanced enough yet to make a contribution.”
"I attend and financially support whatever I can, but have no interest in becoming a developer, so don't feel that my opinions are really considered particularly valuable.”
"Have no idea how I can contribute as not a developer"
As a secondary, quantitative check to balance the qualitative responses, we also asked respondents to rate the importance (on a scale of 1 to 100) of several factors that might relate to their decision not to contribute. These factors were chosen both from the literature on motivations of open source contributors, and from anecdotal reports from conversations I’ve had with others in the community. Figure 3 shows a breakdown of median ratings.
Figure 3: median ratings per motivation answer
As shown in Figure 3, the lack of available time (i.e. “I am too busy with other commitments”) received the highest median rating (80/100) among respondents. However, the next highest median ratings were being unable to write code (72/100) and being unclear on how to contribute (60/100), reinforcing the insight from the qualitative findings that, at least to this sample, knowledge of code is seen as essential to Drupal contribution.
Issues with survey design
As with most pilot surveys of this type, this survey is not without its challenges. First, the sample (as with all surveys) is largely self-selected from my social network, which makes it broad, but limited. Additionally, we found a couple of challenges for people who completed the survey:
- The rating system used for motivational factors (a slider from 0 to 100), while offering some precision, made analysis challenging.
- Selection of a single Drupal “role” was difficult for many respondents, as they see themselves as having multiple roles within the community (e.g. a designer can also be a themer and a site builder). This may have contributed to the low number of “designers” in the survey.
- Questions about respondents’ work life, taken directly from the FLOSS-US study, focused heavily on the assumption that Drupal contributors were full-time employees, which meant that independent consultants found it difficult to answer these questions.
These issues will be addressed in the next version of the survey, which will be distributed again in summer. I’ll be promoting it both at Drupalcon and Design 4 Drupal, so please look out for it!