Developers Corner

Drupal.org Re-Design with Multi-Site Search: Progress in Portland

William O'Connor
CTO

Drupal.org Team Meets In Portland

The Drupal.org redesign team, including recently hired contractors, met in Portland last week to coordinate their efforts and generate a workable project plan.

Among the agreements in place are the scheduling of a development release on August 7th, and the planning of subsequent releases every two weeks after that until project completion.  Additionally, the team will be tracking issues in the public queues and meeting by phone three times each week.

For regular updates go to: http://groups.drupal.org/drupalorg-redesign-implementers 

The hiring of contractors was a difficult decision for the Drupal Association, but it was necessary in order to accelerate Drupal development and to enable the project team to appropriately address the highly complex technical issues involved in the redesign.

Achieve Internet is leading the Solr development portion of the project. Solr is a robust search platform used by many large Drupal web sites to improve search speed and capability. Solr works by housing an index of a site's Drupal data and arranging it for easy access via the Lucene Java search library.

Drupal.org is a huge site, with nearly a million pages, nearly a million users, and at least 8600 projects. Project manager Chris Strahl noted that this puts the Drupal.org redesign project on the same scale as some of the largest corporate website projects in the world.

The redesign goes far beyond aesthetic upgrades to include key functional improvements. The project module, for example, is evolving from a Drupal.org-specific tool to a true project management module with robust Views integration. Also, site searching will include the ability to search all *.drupal.org sites, so the information distributed in various areas (groups.drupal.org, for example) will be accessible via one search box. Add to this increased functionality the implications of the GIT migration, and it’s easy to see how significant this Drupal development effort is.

With such a huge project, no task is truly discrete, and sorting out some of the dependencies has been challenging.For example, one of the nagging issues for Bill O’Connor, Achieve’s CTO, has been the difficulty of running an adequate development environment for his Solr work. Also, since searching is so intimately intertwined with the development of the Drupal project module, it has been difficult to build a Solr index on a moving target.

When the team met in Portland, they were able to address these sorts of issues, prioritize aspects of the project, and lay out clear communication paths for development. The infrastructure team has recently been able to provide a Solr index that can be stored locally, removing a key obstacle to Bill’s progress. And Bill and the project team have created a much clearer plan for ensuring that project field types, for example, meet Solr indexing requirements.

Achieve Internet’s part of the project is somewhat different than some of the others, in that there is less  Drupal community volunteer coordination and involvement. Strahl noted that "Since Bill is one of the few people in the world who can work on Solr for a site of this complexity, he’s hip-deep in Drupal.org Solr development.” Bill’s been focused on 1) ensuring that content indexing enables robust faceted searching and 2) enabling the ability to build dynamic, complex queries in Drupal across multiple sites.

Strahl believes the hiring of contractors and the meeting in Portland have had the desired effect. “More issues have been checked out in the last two weeks than in the last several months combined,” he said. “And the August 7th release will most likely include 50 resolved issues.”

Three of the team members--GIT migration lead Sam Boyer, Bill O'Connor, and Chris Strahl--were in San Diego at Achieve Internet on Friday to participate face-to-face in the weekly scrum, with the others joining by phone for a productive meeting.

“There is no lack of brain power on this team,” Strahl said after the call. “With a team of people who are used to being right, sometimes things can be difficult. But so far communication has been great, and I’m excited about the progress.”

More about Developers Corner

API Documentation, API, Developer Portal
Below is a comparison of the features SmartDocs and Swagger UI offers out-of-the-box for the Drupal-based developer portal. The information here is to highlight the difference between two commonly used modules and to make it clear what could work best for your business.   
  Welcome Drupal 8.6! More change to the Drupal-verse landscape, and more new features to look forward to. This will obviously affect all current D8 users, but what should you look out for? Let's examine how it might impact your site and what unexpected issues may occur.