Moving from Facebook Groups to Discourse

The following is a guest post by Martin Eriksson @meriksson

Update: Unfortunately due to changes to Facebook's API it is no longer possible to extract Facebook data for migration. If you're considering using Facebook Groups as a platform, check out our article on the pros, cons, and alternatives to using Facebook Groups.

On January 4th 2017, the news aggregation site cor.ax completed its migration from private Facebook Groups to a private Discourse community.

Why the move?

We used to have about 20 Facebook groups for people involved in a network of alternative media projects. Some of the groups were dedicated to editorial discussions, some were general discussion groups for our paying members, some were interest groups about financial issues, gaming, cultural topics etc.

We started using Facebook since it was very easy but we were never comfortable with it. As our network grew and the groups become more numerous and active we become more and more worried about key parts of our infrastructure being under the control of a third-party. If we lost access to our groups it would have caused massive problems for us, including potentially decreasing revenues.

In particular, we were worried since Facebook has increasingly been shutting down not only public pages but also private groups – at least, so we have heard. Since our project has to do with non-mainstream politics and activism, we did not want to take any risks with possible future purges of groups perceived to promote subversive ideas.

Another recurring problem for us was being unable to bring new participants into the project due to them not being Facebook users. One typical case was someone being interested in becoming one of our volunteer editors, but being rejected due to not being able to follow the crucial editorial discussions in the Facebook group for editors. Another, and increasingly costly, problem was people being reluctant to sign up as paying members since our community forum was based on a platform they did not want to use.

After making the move, we have indeed seen an influx of new editors as well as paying members. And not least, we now have people making good contributions to discussions who previously supported the project financially but was on the outside of the social parts of it. Dozens of people actually created Facebook accounts, or reopened previously closed accounts, for the sole reason of accessing our groups. This was something I was personally very ashamed of. Happily, many of these and others as well are now shutting down their accounts or drastically decreasing their activity level on Facebook.

How we made it happen

When we began the migration project I started to look around for ready-made scripts for exporting from Facebook and importing to Discourse. I found one but it was rather rudimentary or experimental. So I realized that we would have to develop a solution ourselves. Luckily I happen to have a background as a Ruby programmer so in part it was a fun and stimulating project for me. And having the existing basic script as a starting point made it really easy to get started.

Some of the major obstacles were working with undocumented aspects of the Facebook API – it turns out that there are many! For example, it was often impossible to access particular objects without any explanation given. In part I had to work by trial and error to figure out how things worked. There are also inconsistencies, most of which are noted in the importer description.

Besides purely technical problems we had to think carefully about how our 20 existing groups could be mapped to the Discourse model. Should we use one Discourse instance or multiple ones? We considered having one Discourse instance per language (we have two languages in daily use among members) and also having one for the editors who run the core projects and one for our community of paying members (readers, podcast listeners etc).

In the end we decided to go for one single Discourse for all groups and instead created an intricate system of categories. One of the reasons for this was to enable searching through all the material instead of forcing users to search on multiple sites. In hindsight this was certainly a good decision, but it created some problems due to the Discourse model being – according to my personal impressions – more oriented toward relatively open forums rather than rigorously segmented ones.

Each Facebook group was imported to a single category to begin with, but since Discourse gives us so much more flexibility than Facebook we could then immediately begin restructuring things. A couple of weeks later we had about 40 categories and more than 10 groups to control access to them. One of the primary groups gives access to about half the categories. This is the group for our paying members who get access to a large number of discussion groups - previously this was very complicated to maintain as a set of Facebook groups, since new members needed to be added individually to each group and we did not know what groups they were interested in. Using categories and groups in Discourse makes it much easier.

We are not done reorganizing all our imported topics in the new categories; a handful of moderators are working on it but we still have more than 2,000 topics in a non-specific category. As more topics get organized, the forum becomes better and better. However, since Discourse is not really intended to be used like this (i.e. doing intricate manipulations of historical content) we also have a number of quirks still be being worked out. We're actively engaged with the meta.discourse.org community, discussing our issues in some new, some existing topics.

The perks of a Discourse Community- People do not need to have Facebook accounts to be active in the community.

• Search works really well in Discourse (except for being limited to 50 results per query). The search feature in Facebook groups is badly crippled. It seems Facebook wants to steer users away from discovering old threads and instead get them to start new ones. This breaks up interesting discussions in ways that are very bad for exchanging ideas, including as many users as possible, having long-running discussions etc. Needless to say, if you have multiple Facebook groups, there is no way of making one search query for all of them. Moving to Discourse has given new life to thousands of threads which have high-quality content but was previously in effect hidden by Facebook.
• The Discourse model for threads, replies, quoting etc is a far superior way of structuring conversations. This really comes down to Discourse having a very good and thought-through model whereas Facebook optimizes for entirely different things than serious discussions. It took some time for people to get used to quoting but by now discussions are much easier to follow.
• Access management through Discourse groups and categories is far more convenient than managing membership in Facebook groups. Also, SSO integration removes the need for adding and removing people for groups. Once members pay the membership fee and get accounts on our site, they can access the forum without someone needing to add them to groups, figure out which of our groups they are likely to want access to etc.
• The category system in Discourse makes it a lot easier to create a customized structure, it can be changed easily later on if we change our thinking about it etc. In practice, the forum feels much more like our own which creates an atmosphere if familiarity and trust. Along the same lines, all the detailed customization options makes it feel much more like a true home for our project, rather than some space we borrowed from someone.
• Mail notifications are way better, from customizable levels of following/watching to summary emails. Related to this, tagging works very well and we can for example ping an entire group (normally generating mail notifications to each person). On Facebook we used tagging a lot but it was very cumbersome and time-consuming and there are several limits (only 50 users can be tagged per post/comment, there is an additional limits on how many tags a particular user can create per day – of course, Discourse also has such limits, but we can customize them ourselves).
• Activity statistics are great for managing a community, both for admins to keep track and for users to see how their engagement compares to other users. Facebook has nothing similar to this. There is a tool called Grytics which can do similar things but it is cumbersome to use and in any case inferior to the built-in Discourse tools. After importing all our historical Facebook data into Discourse it was amazing to be able to see aggregate statistics on e.g. total number of likes received and other per-user breakdowns.

Many thanks to Martin Eriksson @meriksson for not only sharing his migration experience, but also the code used to accomplish it, which is shared freely on GitHub.