Introducing the Community Moderation system
July 6, 2023
There’s now a new moderation system in place! Let me explain how it works:
When you submit a new activity now, it will be visible only to you (you can find it by going to “Your activities” on the Dashboard) and a new group of users called Community Moderators.
A Community Moderator is a user who has volunteered to help with moderation. They can visit a page on the site that lists newly submitted activities. When they look at an activity, they don’t see the user who submitted it, but they see its regular contents like the description and file attachments. They then choose whether:
- The activity follows the rules of the site and is okay to post.
- They would like the user to revise the activity in some way.
- The activity should just be deleted. Maybe it’s spam or it just doesn’t seem salvageable.
The moderator can write out the reason for their decision and the user can see it when they look at their activity.
When the moderators have reached a consensus, the activity will either be approved or deleted. A consensus is determined by how many moderators are currently active at any time – if there’s an even number, it’s half of them (so 2 moderators out of 4 will need to make the same decision) and if it’s an odd number, it’s a majority (so 3 out of 5 will be a consensus).
Community Moderators are appointed for a limited period of time – my initial goal is for it to be for terms of something like 2 or 3 months. To prevent them from catching any grief, they’ll be anonymous – their feedback for an activity will be under a 4-digit code number like “234B” which will be present for their term as moderator, but then retired after that. Also, Community Moderators can’t see the feedback or decisions of other moderators.
I know this is a little convoluted, but my goal with this system was to make a moderation system that involved a group of people doing their best to render an objective decision on new activities. Also, the limited time nature of each moderator’s tenure will hopefully avoid cliques developing and bring in a wide variety of perspectives to the moderation decisions.
I’ve appointed an initial group of moderators from some users who have contacted me recently and volunteered to be involved in moderation. In the next month or two, I’m going to make a form that you can submit on the site to request to be a Community Moderator, so ideally I'll be able to draw from interested users to have a rotating group of moderators at any given time. It will be possible for users to be moderators multiple times, but I’m hoping that there will be a healthy mix of new users as well.
I should mention that this only applies to activities at the moment. I’m planning on using this system for comments as well, but they function different technically (and you can’t edit them) so it will take some more time to code that in.
There’s still a lot of this system that I have to refine, but it’s in a state where I wanted to get it up on the site and see how well it works. Even if all the things that happened in the last few months didn’t happen, it’s a system that I wanted to implement to help free up some of the time that I’ve been spending on site moderation. I really want to harness the power of a lot of people working together for the goal of making the best resource possible. I have a few more things that I want to try in that vein, and I think this is a good step in that direction.
I anticipate that most activities will probably continue to be approved in this system, but I hope it will also provide people with useful feedback on their activities. At some point soon I’ll add functionality so that even after an activity is approved, the uploader can see the Community Moderator feedback for it.
Please let me know what you think, or if there are any issues you encounter!
I think this is a pretty solid starting point. Thanks for putting in the effort to help this community not devolve into chaos.
Thanks for your hard work Jake! Looking forward to seeing the outcome and moving the site forward into an even better experience for everyone.
awesome thanks
Seems like a great implementation of a moderation system without the risk of bias due to username (from both sides) or post history. I'm looking forward to seeing it in action.
This is a great starting point for sure! Thanks so much for your love for this community! I'm looking forward to seeing this site flourish even more in the future! :D
Sounds like a good system. Thanks as always for your hard work.
Thank you for putting so much effort in Jake. Thank you for being so transparent on all the updates and issues.This is a good start but hopefully the activity sent wont take long to get approved. Sometimes, we have limited time to come up with new ideas and want to see new activities to try with our classes. Thank you so much !
"They would like the user to revise the activity in some way" Is something Id like more Clairity on how that will be done? Will moderators be offering suggestions they think will make an activity better? Or just making sure an activity is in line with the new guidelines for the site? I love the wide variety of ways people do activities, so im hoping this doesnt somehow lead to activites becoming less diverse. Maybe im just misunderstanding the wording. Thanks!
When Moderators make their decision, they can leave some feedback for the user. It's about the same size as a comment. So they can write something like "Can you revise this to include ...?" or "Please change the title to..." and the uploader will see all of their feedback at the same time. Moderators can revise their decision later if the uploader edits the activity.
Sorry, and some additional notes: The approval process should happen fairly quickly as long as the majority of the active moderators are checking the site fairly regularly. To take some of the load off of the server, the system checks every couple of hours for activities that are unmoderated and does the vote counting check. There have already been several activities that have been approved today, so fortunately that means that at least some part of it works!
I ask the moderators to make their decisions based on the site rules, but I think it's natural that there will be some degree of personal preference or personal interpretation of the rules as well. I think that trying to impose complete neutrality would be almost impossible, so that's why I thought it would be good to have several people doing it at the same time, and bringing in a mix of people over time.
This sounds like an excellent, fair, thorough moderation system. Well formulated and implemented. Thank you so much Jake!
This seems like a near-perfect system (at least as close as it is to get to one). Thank you for your efforts!
Nice work. Sounds like a great system to me too.
This is a great start to any issues that occur with activities. It sounds reasonable. Thank you for your hardwork on this site Jake.
It looks a little rough at the moment, but you can go to the activity controls for one of your activities (the gear menu) and click an option that says "Moderation history." For most activities posted before the last week this will be empty, but if you have a new activity that's gone through this system, you can see the moderator feedback even after it's approved.
I hate to bring it up, but have you thought about a user changing an activity so drastically that it breaks the site rules once it has been approved?
Since moving to Elementary School, I've had to use ALTopedia less because I've been able to pull lessons from my own past files, so I was somewhat insulated from what seems to have gone down. It's a shame that this system became necessary.
That being said, I'm glad to hear of the strong focus on anonymity, both by keeping the submitter of the activity private from the moderators and keeping the moderators' identities private among themselves and with the public. I hope this system works, because it seems well-designed.
Hopefully the short tenure does keep cliques from forming. Although the possibility is remote, if moderators do figure out who the others are, some could feel pressured to approve activities they don't think should be in order to make a majority vote.
I haven't had time to work on it yet, but I'll need to do something to address situations where an activity gets approved and then edited drastically enough to need re-evaluating. A long time ago, the system used to pull activities from public visibility every time they were edited to be re-approved, but that became unfeasible as the site got bigger. A lot of people edit their activities to clear up the description or add files later, and 99.9% of the time it's not a problem and isn't worth asking a moderator to check.
Activities can now have multiple moderation histories, so they can be pulled from public visibility and re-evaluated later if necessary, but I haven't coded in the system that pulls them yet. A lot of users were asking for a report system, and while I need to build one in at some point, it's something that I'll have to be careful about because it could generate a lot of spurious or petty reports. For now, I can manually pull activities if needed.
As for moderators knowing about each other, it wouldn't be the end of the world, and I don't know if the stakes are high enough on anything to make moderators want to coordinate with each other. If a moderator wants to say that they're a moderator, then I don't see much of a problem with it. I built the system to allow them to be anonymous if they want to, in case anyone gets upset at their decisions.
Perhaps all the activities on the site could automatically come up for re-approval after a year or two. Besides keeping everything updated with the site's rules, it would help older activities stay updated with more recent information, technology, history etc.