blog

archives

OpenROV and Discourse

Jeff Atwood June 15, 2015

We measure Discourse’s success as an open source project by one simple metric: are Discourse communities working? Is your community working? We recently interviewed Brian Adams of OpenROV to discuss how they use Discourse and get some specifics about what is – and isn’t – working for their community.

What is OpenROV?

OpenROV is both a product and a community. The product is a low cost yet highly capable Remotely Operated underwater Vehicle that we sell as a kit. It is all open source hardware and software that takes a weekend or so to build, but then allows you to pilot the ROV from your laptop as it streams HD video from up to 75M (246ft for us yanks) below the surface. The community is international, in 50 countries. It includes makers, explorers, treasure hunters, educators, college researchers, government agencies, and programmers.

By the way, for the programmers out there: it’s an underwater robot with lasers that runs Node.js, Linux and Arduino! Get involved :-)

When did it start?

OpenROV started back in 2010 with Eric Stackpole and David Lang. Their goal was to make a super small ROV that could find treasure buried deep in a water filled cave. They decided to open up the idea to anyone who wanted to participate and created an online forum where all kinds of people came and pitched in with ideas to help create the ROV. In 2012 they had a very successful kickstarter campaign to make the kit available to everyone. That kickstarter put us on the map.

When did your community form?

The community came at the very beginning, so that makes about 5 years of ongoing discussion on underwater robots! Here’s our first post.

How did your community connect?

I can’t even imagine trying to engage everyone without some type of Forum or BBS system.

By January 2015, you decided to migrate to Discourse. What prompted the move?

We started out with Ning when we began. Back then Ning was still hot with lots of funding behind it. As an out of the box community building solution it was great. After five years, Ning changed direction and were no longer investing in the forum and other community tools. The lack of innovation and focus on community building was a real problem.

With our Kickstarter behind us we were heads down on getting the ROV in the hands of as many people as possible. A healthy and vibrant community that offered both support and encouragement was key to our vision of how to grow. In our hearts, we knew the longer we delayed in cutting the community over to a new forum solution, the harder it was going to be. So in January we took the plunge.

You moved a 5000-member community to Discourse. What was the (technical) migration process like?

In our case we had a couple key technical hurdles to over come in order to move to Discourse.

  1. The scariest for us was to migrate all our old data from Ning. We had some true gems worth of data and discussions that reflected the identity of our community and there was no way we could afford to loose that. Neil from the Discourse team was our primary liaison for the migration. My only job at that point was to get the data out of Ning. Fortunately there was an export tool that kinda worked by dumping a huge JSON feed as well as most of the assets (video and photos) from our old forum. Neil took that export and created a sandbox site for us to login and review. The sandbox site was great, it allowed us to customize all aspects of the forum before go live with our actual content. We used the sandbox to find the edge cases that failed in the migration, and Neil would make updates and reload the data set for us to review. The only thing that we lost in our migration was the activity, number of reads and likes. Turns out that data just physically did not exist in the export files.
  2. The second major hurdle for us was Single Sign On. Ning was a service that integrated a blog, forum, photo gallery, and a ton of other mini apps all in to one monolithic ecosystem. Much like WordPress with all of their modules. Its easy to take for granted that the system in sharing a single authentication between all of these applications. When we chose to go with Discourse, we were choosing to integrate a lot of best of class services, forum, blog, etc… instead of getting a single integrated solution with a bunch of not best of class services. Integration for us meant a single identity that would transparently sign you in to all of the services. We ended up choosing Auth0 as our authentication solution. When I had initially gone through the meta forums on discourse, I had seen there was an Auth0 plugin available. But as we got in to the migration to the Discourse hosted solution, I ran smack in to the fact that the standard and business hosting plans don’t support customer managed plugins. No Auth0 plugin for us. Well Auth0 was made to support all of the standard authentication protocols such as OAuth and SAML… but Discourse used its own proprietary SSO solution around javascript tokens. In the end, I had to spend a weekend bringing up a bridge service that would translate between Discourse tokens and Auth0 tokens so that my users could authenticate to Discourse now and then other services in the future.

The rest of the technical parts of the migration were very straightforward. We ended up with a cut over night, took the forums down for a few hours, ran in the latest data dump from Ning, swapped DNS and we were back.

Migrations present human challenges as well as technical challenges. How did that go?

We knew at the very beginning that we might lose people in the cut over. We accepted that and did our best to mitigate that as a risk. There were several discussions on the Meta forums where others had shared ideas. I read all of those discussions. We ended up with a blog post to our community explaining what we were changing and when we were going to cut over. We shared links to both the sandbox test site at Discourse as well as to several other large communities that were using Discourse.

At the point of the cut over we also posted multiple pinned reminders in the forum explaining how people would need to re-register their accounts to re-link to their old data. That last issue was probably the one we could have improved the most. After the cutover, for the next few weeks, I was monitoring the logs in Auth0 and Discourse to make sure people were able to log back in to their old accounts. Turned out it was just too confusing. I could literally see people try over and over to reset passwords and login without verified emails addresses over and over. All of the help text around the login pages was just not enough. In the end I finally figured out how to export the user list from Discourse (that had been migrated from Ning with username and email, but no passwords) and then upload that data to Auth0′s APIs. Now people could simply use password recovery to log back in to their accounts vs. having to re-register and revalidate their email address.

Usage right after the cut over was a bit flat. Now that we could see the metrics, it was clear that a lot of our community had slowly been becoming less active well before we moved. One of the things that Discourse offered that I really liked, was a sophisticated email notification system to keep up engagement. It turns out that these were not working for users that had not logged in to the new forum system. It took a few weeks to figure this out, but once we did and Neil corrected the status data for our users, we saw 1700 summary digests fly out of the system which really helped get people to come and see what was going on. You might note that 1700 is very different that 5000. As it turns out, that was the number of users in our system that had been active in the prior system in the last year. Did I mention how nice it is to have the dashboard metrics in Discourse? In our old system, to cut down on spam, the co-founders were manually approving every new user request. We found that a lot of account activation requests were lost to the spam god ether. We ended up removing an embarrassingly huge number of accounts that had never been approved. I’m so happy that the anti-spam solutions in Discourse mean we no longer have to approve every new user by hand.

As far as uptake with the Discourse forum software goes, it’s still evolving. Our community is still fairly reactive to how much energy we put in to it. On days that we engage in the conversations we see a lot of traffic with people jumping back in to the conversations. I think the email digests have become just something that people expect, which is good – I should probably ask for a report to figure out how many people if any have changed the digest settings. For most of our users, I think the data entry has been intuitive, no complaints.

What’s your favorite type of interaction on your forum?

By far my favorite topics are those community members share about their own expeditions and ROV projects. When I see someone post in our “build report” or “projects” categories about robotic hands, an all metal OpenROV variant, or spotting whales, that really validates what we are trying to do.

You also have a dedicated support desk. How do you balance these two methods of communication?

As I mentioned, we are a community, but OpenROV is also a company with products that we sell. We spent a lot of time trying to figure out the best way to support both. We truly believe that for most questions people have about our product, they can find it by searching the forums. If that doesn’t work, they can get their questions answered the fastest by posting in the forum… and help the most people out because the question in public.

At the same time we had to acknowledge that not everyone was comfortable publicly discussing issues they might be having. So we ended up with a compromise. We list the forums first in our support section, but we do have a contact support option which will open a support ticket in Zendesk. Our customers are going to use whatever they want. Those of us that moderate the forum help direct users to the Zendesk support if we don’t think the forums will help (such as for shipping/ordering questions or warranty support) and the support team will often recommend a user back to existing answers in the forums.

We tried to setup an integration between Zendesk and Discourse, so that moderators could tag topics so that they create tickets automatically in Zendesk, and so that our support team’s responses to the Zendesk ticket would automatically be attached back as forum posts. Unfortunately we could not find a way to make that happen. Were hoping that future Zapier (of IFTTT if it ever happens) support might help us glue to two products together.

You’ve had nearly six months with Discourse, what’s working well for you?

Metrics, notifications, email replies to topics, oneboxing of links, search.

Discourse is open source and evolving rapidly; what are your most wanted improvements as a hosted customer?

A few things:

  1. Editor improvements to expose the additional features such as detail sections, polls, strike through, etc.
  2. Ability to let users select their preferred styling (specifically layouts). I find some users have a hard time with all of the metrics, while others, especially those that post a lot, want to see how much engagement they get off of a topic.
  3. Click through indicators on embedded videos.
  4. The Solved plugin that is in the works.
  5. A photo gallery plugin. There are topics where you ask people to post their best picture of X. The main topic should have a carousel to move through the photos.
  6. Markdown table support. I read the posts debating that. Fact is we post all kinds of side by side comparisons… oh wait, we can’t. :P

You’re using a custom style for the topic lists. Can you elaborate on why you customized here?

My problem with the default styling is that once you spend enough time with it to finally appreciate it, you forget the initial gut reaction you had to seeing it for the first time. In my experience that was “busy”. The call for action was buried. Eric is our co-founder who started the community. After the roll-out, Eric re-lived those same gut reactions, but was so busy with other things that he has not spent enough time to really get the forum software. Eric is the heart of our community and removing every barrier to his keeping active is a huge priority. Simplifying the screen so that he gets engaged enough that he would appreciate the “expert” mode layout is one of those tactics. Thus my issue #2 in the wanted feature section.

What’s your fondest memory of forums, way back when, or recently?

When I was a kid, I was rather involved in the BBS scene. I had a friend that ran a site (on a Commodore if I recall). Anyhow, when I signed in I used a lot of fake stuff in the registration form… at which point he broke in to the session with “Hey Ye bullshit artist…”, and a handle was born. Back then we spent a ridiculous amount of time making ASCII art photos for signatures. I still remember the one I used. Brings a smile.

╘══▒▓▓▓▒══╛
    ▀Ω▀

Thanks Brian for these detailed responses. For more information, visit OpenROV and check out the OpenROV community!

4 Replies

Discourse 1.3 released!

Jeff Atwood June 3, 2015

With the help of our community and our customers, we’re proud to release Discourse 1.3 today!

The focus of this release was new features, as requested by our customers and users.

Anonymous posts

For sites where the subject matter demands that people sometimes post anonymously, we now support an anonymous mode, if enabled in your site settings. Simply select Anonymous Mode from your user menu and you’ll be provided a new anonymous account account to post as. Switch back and forth to anonymous mode as much as you like.

Desktop notifications

Discourse now supports the new WHATWG notifications API for desktop (and mobile, in some cases) notifications! This means, depending on your OS and browser combination, you may see small banners from your operating system notifying you of events in Discourse as they occur. You do need to opt in when asked by your browser — you’ll be asked the first time a notification occurs. Thanks to Kane York for all his hard work on this!

Civilized Mute

We’ve always been opposed, for both social and technical reasons, to the idea of muting a person and making them entirely “disappear”, but community member Accalia proposed something more civilized — what if we just muted all notifications from people on your mute list?

That did indeed seem rather civilized to us, and minimizes any disruptions to yourself and the community. So you’ll find a new muted user section on your user page.

Better Polls

We launched with a basic poll implementation in Discourse contributed by the community, but it was high time we improved that. So Régis rolled up his sleeves and made it awesome and way more flexible. We now support single or multiple choice polls, numeric polls, polls can be in any reply in the topic, and you can even have multiple polls in the same post.

The simplest form is now:

[poll]
- Option A
- Option B
- Option C
[/poll]

Post Approvals

One of the commonest requests was for a mode to approve all new user posts. This is especially important if the discussion is based on a sensitive or potentially dangerous topic. We now ship with a mode where staff can manually approve the first (n) new posts from every user who is below a given trust level. It is a bit of an inevitable tradeoff of staff time versus user time, but once post approvals are enabled you can rest assured that new users won’t come along and cause problems in your community while they learn the ropes of posting.

Cleaner default style

Discourse is built around the idea of a simple, minimalist design, but we haven’t always achieved that. In Discourse 1.2 we tried to de-rainbow a bit with less emphasized category badges, and that simplification trend continues in 1.3 — except we’re also trying to de-grey the design and make it cleaner, airier. You may notice grey tiger striping is gone, the intensity of grey quotes is turned way down, and expanded quotes and replies no longer have grey backgrounds.

There was, uh, a lot of grey. Look how far we’ve come from Discourse 0.9!

User Directory

We finally built a user directory at /users which lets you see where your community fares on a variety of sortable metrics, including read time, posts, and days visited.

Simpler Pins and Banners

We unified the UI for deciding how you want to “feature” a topic, from pinning in the category, to pinning globally, to making a site-wide banner. We also let you know how many pins you have so that you’re not overwhelming your users with static information.

More Flexible Invites

Because discussions are more fun with more people, we continue to improve and enhance our invitations system. By popular request, you can now invite existing users to a topic, not just new users via email.

More Code Language Highlights

If you post source code on your Discourse, you’ll be pleased to find out you can now customize exactly what programming languages are highlighted in the code. No matter how common or obscure, it’s probably supported by highlight.js, so just add it to the list in your site settings!

Improved Importers

If you’re thinking of making the switch to Discourse, it is easier than ever, because we now have open source importers for vBulletin, phpBB, Vanilla, Ning, Kunena, Discuz!, bbPress, Simple Machines, Drupal, and myBB.

You’ll still need to be a developer to perform the import, but this is something we plan to make easier by the end of the year.

Better Performance

You should see better performance on the client with our Ember 1.11 upgrade which offers a 30% speed boost. And on the server, we made several enhancements for large topics and large sites to keep things speedy.

And so much more

These are just the major highlights in 1.3 — there are literally hundreds of other tiny improvements, refinements, and bugfixes in 1.3 that we aren’t covering here, but are in the full release notes.

Easy One Click Upgrade

I think you’ll agree that 1.3 is the best version of Discourse EVER!

Upgrade your Discourse instance today to get these amazing new Discourse features (and OK, a lot of some bug fixes) via our easy one click admin updater linked in your dashboard:

Thank You

As usual, we’d like to thank our customers for your support, and the broader Discourse community for all their contributions toward this release — whether it was in pull requests, feedback on meta.discourse, or our personal favorite, feedback based on user activity in your own Discourse instance. Thank you!

(Also, a particular thanks to avid meta.discourse user erlendsh who created the super cool feature demo videos and screenshots that you see above.)

For insight into what’s coming up in future releases of Discourse, keep an eye on the releases category at meta discourse. Keep the feedback coming! We’re always listening… although we listen especially hard when you purchase a hosting contract with us. ;)

16 Replies

Welcome Arpit Jalan to the Discourse team

Jeff Atwood May 24, 2015

It’s been almost two years since someone joined the Discourse team, but we’re speeding things up this year as our hosting service expands.

Today we’d like to officially welcome Arpit Jalan from Jodhpur, India to the Discourse team.

Here’s what you need to know about Arpit: we did not pick Arpit. He picked us.

We discovered Arpit through our meta site where he just … started building things in early 2014, like the first easy Discourse install guide. We were impressed, and when he wrote us in May 2014 asking for any kind of opportunity to work together, I can’t say I didn’t see it coming.

What I didn’t see coming is Arpit’s depth of conviction. I can’t remember the last time I read such an impassioned call to action by someone who feels they are at a critical juncture in their life. It’s really hard for me to do it justice by describing it here, but at the end of it he cites the 1993 movie Rudy as an inspiration – another guy who has all the heart in the world.

We started with Arpit on a year-long internship plan, and his work has been consistently great. You may recognize Arpit’s hand in …

  • Big improvements in our invite system
  • The ability for any Discourse user to export their posts
  • Managing our Transifex translations
  • Many Discourse howto guides
  • Our internal site monitoring script for hosted clients

… among many, many other things that he did to improve Discourse in the last twelve months.

So when his internship year was up, of course Arpit “Rudy” Jalan made the Discourse team.

You might say he’s been ready for this his whole life.

Welcome to Discourse, Arpit. The work is hard, the hours are long, and our mission will take a decade – but I can’t think of anyone else I’d rather have on our team.

6 Replies