The news mod team has asked to no longer be a part of the project until we have a composite tool that polls multiple sources for a more balanced view.
It will take a few hours, but FOR NOW there won’t be a bot giving reviews of the source.
The goal was simple: make it easier to show biased sources. This was to give you and the mods a better view of what we were looking at.
The mod team is in agreement: one source of truth isn’t enough. We are working on a tool to give a composite score, from multiple sources, all open source.
Rule 1 Ban Count: 5
- @LagrangePoint@lemmy.world (Permaban)
- @aniki@lemmings.world (15 day ban)
- @catloaf@lemm.ee (15 day ban)
- @DoctorButts@kbin.melroy.org (Permaban)
- @stormesp@lemm.ee (15 day ban)
Not a good look, mods.
I’ve been busy with work. I’ll review the bans. I can say that none were banned by me.
https://a.lemmy.world/lemmy.world/modlog/view?target=4123405
Comment that caused the ban is in the logs
https://a.lemmy.world/lemmy.world/u/stormesp@lemm.ee
No comments removed, but running theme was attacking mods (well, me). Yes, we’re mods, and I don’t take action against people arguing with me, but the other mods will enforce the rules evenly.
https://a.lemmy.world/lemmy.world/modlog/view?target=6038477
Reason is in the logs, and aligns with community rules
https://a.lemmy.world/lemmy.world/u/aniki@lemmings.world
Reasons aren’t noted, but ready to see when you read their last few comments in news
https://a.lemmy.world/lemmy.world/modlog/view?target=3325340
Reason given, and valid, in the logs.
What I discovered: people got pissed at the mods for wanting something, and our reasons be damned.
What I’m making isn’t a bot, but an API, but everybody got pissed I wanted ANYTHING that would help detect false news.
I even got rid of MBFC, but that’s not enough. I get shit on for making an open source tool, unrelated to the bot.
You think that’s all legit? I don’t see permaban levels of rule breaking at all there. More admin and moderator boneheadedness causing mild irritation with users. You’re asking for feedback, then ignore most of that feedback, and now ban disagreement because they’re understandably frustrated with your responses.
Like I explained to you yesterday, none of this feedback has been ignored. We took 12 days to review it, and we acted accordingly.
What I’m making isn’t a bot, but an API, but everybody got pissed I wanted ANYTHING that would help detect false news.
That’s not really clear from your opening post. I don’t mean that to be critical, just saying the opening post sounded very much like the pending release of bot v2.
Fair enough. It’s been a busy week for me at work.
The news mod team has asked to no longer be a part of the project until we have a composite tool that polls multiple sources for a more balanced view.
Thank you. It is often difficult to change course once it is set. I appreciate the !news team reaching out to the community and acting on their concerns.
Keep it gone. MBFC and others are not a source of truth.
Adding multiple sources of bias does not produce an unbiased result.
2nd. It’s just not necessary. Frankly what’s more off putting is the outright bizarre insistence on the mods part and outright denial of feedback. If the bot comes back I’m blocking the community, there are other streams for news.
Wouldn’t it be easier to just block the bot?
If everyone who doesn’t like the bot blocks it, people entering the community will see the bot upvoted. That will mean they assume the general consensus is that the bot is trustworthy and accurate.
I agree that the bot is problematic and don’t think it provides valuable information. I personally have it blocked. Leaving the community and blocking the bot both result in one less person advocating against the bot.
I disagree. I often want to check the reliability of sources. And a composite would be great, because MBFC is obviously heavily flawed.
Ok. Think your response through.
That means all news outlets are biased as well. This is why we want something that gives a composite score. If all sources say “this news outlet is shit”, maybe we take it with a grain of salt, or maybe we black list it. At a minimum, it helps mods and readers get a context of the content.
LW admin/mod team seem to have this overbearing and weird belief that they need to tell everyone else what to think and how to think it.
How about… you all just fuck off and don’t?
I’ve thought it through. We should not outsource critical thinking and media literacy.
I think the problem arises from the fact that I don’t know what you mean by “this news outlet is shit”. Maybe we can define exactly what we mean here and block such news sites from being posted.
I don’t think bias can be correlated with article quality, and we should be engaging with articles and ideas based on the merits within, not some aggregate made up thing like “bias”. I’m not saying it’s not a real thing, just that it’s made up and subjective enough to be in my view a useless measure and a fruitless endeavour to get a meaningful measure in the first place.
If you want a bunch of opinions on the usefulness of an article then we have votes already.
Obviously I don’t have the context of a mod, so if there are specific cases where you need a bias rating, however flawed, to do that job effectively then sure but I think that’s best developed as say a browser extension (or maybe one exists already) so it’s at least opt in.
EDIT: Also want to say I appreciate both the call for feedback and also the decision to opt out of the bot for now.
Sorry for any confusion. We’re moving away from bias - that’s the goal, at least.
News source being “shit” examples:
- A Voice for Men
- The Activist Mommy
- Adams County Times
- Akron Reporter
- Albany Standard
- American College of Pediatricians
- Ames Today
- Antelope Valley Today
- Baltimore City Wire
- Benton Times
- Bloomington Leader
I could go on, but I’m at work right now.
Can you tell me how many articles from those sources were shared the last month? Because i havent seen many or any at all, but meanwhile i have seen Al jhazeera and other news outlets being called not credible without much basis. If the information shared on an article is not factually correct, you can write a message or edit the title so people can know it or even remove the thread if it can be harmful, but what the mod team here is doing is just absurd. The goal should be to get rid of the bot already.
look at the mod logs for removed posts.
Right, so remove the shitty ones and don’t bother everyone else with it?
Thank you for confirming that having a bot leave a shitty opinion about how trustful that source is or not is not needed. Unless you are saying that the threads removed due to not having a reliable source is based on mbfc, which would be extremely funny and sad at the same time.
Glad to hear we’re moving away from bias, I didn’t pick up on that which could have been my fault so appreciate the clarification. All the best with your work day.
I’m absolutely for the bot and I know I’m not alone. I like having it and I find it useful. I don’t know why other people think it’s “a source of truth” like I’m some mindless sheep who can’t think on my own. I can and do take its rating with a grain of salt.
I don’t like sports but you don’t see me asking admins to remove those subs. It’s selfish of people to ask for it to take it down for everyone. A good aspect of using Lemmy is being able to customize your experience–so do that. If you don’t like bots, hide them all in the settings or block them individually. It’s that simple.
Now that I think of it, maybe Lemmy should ask new users how they want to experience the site when creating their accounts.
Because a lot of people don’t spend effort researching the sources. When the mods present MBFC ratings on each post without any explanation or context, it’s an endorsement of MBFC and their opinion, and presents it as a reliable source.
If people want to do the research to evaluate a source, they can do that on their own. Presenting a biased source like MBFC is counterproductive to that goal.
Because a lot of people don’t spend effort researching the sources
I don’t think that’s fair or true, especially on Lemmy. We don’t need babysitting. And even if it is an endorsement, what examples of reviews do you have that call MBFC into question? I say we don’t bog down ourselves with whether they labeled something center-left that may be center-right or whatnot. What matters is the facts. Everyone can have a say in how they interpret whatever slant they find. Nobody takes it as the ultimate judge. We don’t need it to be impeccably accurate and perfect, either.
In my case, I pay little attention to the rating on sources I already trust. Instead, I use it to hold obvious propagandists to face the quality of their posts. I have many instances of that happening. There’s value in this.
If people want to do the research to evaluate a source, they can do that on their own.
This is one way to get started doing that. It’s a convenient shortcut for the search I was going to do anyway. I’m surprised you preach about source evaluation and push back on this in a post about allowing for multiple sources.
What I wish we had is a tool for showing which sources tend to be most statistically correlated with each other, without trying to place them on a linear spectrum.
Can you give me an example? I may be able to code it
Interesting, almost sounds like a graphdb + magnitude project
Cool stuff
I was thinking of something like the graph of subreddits from this paper—although I think that’s based on subscriber overlap, and I don’t know if there’s a similar metric that would cover all news sites.
I don’t see an easy way to accomplish this without either pulling in the full text of every article over some period and running something like paragraph/doc/site vectors and then clustering by site vector.
That’s putting a lot of faith into unsupervised learning, and it’s probably just as likely to pick up on stylistic conventions like byline and date formats as it is to cluster by some common thematic pattern like political leaning.
Maybe you could use a source site’s posts and upvotes in different fediverse communities as a proxy (assuming you could find representative communities with a similar range of biases).
That’s…actually not a bad idea. Take the user-domain name pairs and weigh the edges between domains by the number of unique users who posted from both domains.
For producing clusters from the resulting graph should be easy, but aside from just saying “these are similar websites” does it really say much?
You could do something similar with comment/upvote/downvote based linkages - maybe they’ll have some deeper semantic meaning
I liked it. Even a biased MBFC that is consistent in it’s bias has value, as you can take the bias into perspective on interpreting the rating.
Honestly; I think the “Negative” reactions to the bot are overblown and only done by a vocal minority who are sockpuppeting followed by a few people who are irrationally angry that the bot can be, GASP! Dare I SAY IT???!!11, Wrong.
Personally I don’t find the bot problematic at all; and I think it could easily be blocked or ignored by people who find it too inaccurate. So I find it extremely disappointing that the mods are listening to the vocal minority about this.
That being said; I do understand why Mods want to make the bot more accurate. It’s assessments and information can easily make obvious extremists and trolls more obvious to the naked eye; and can help people consume media with some grains of salt. More sources of data are good for accuracy.
What point does a “bias” bot serve if it can be incorrect? And if it can be incorrect then why should we trust it at all?
You may as well write a bot that posts “remember, don’t trust everything you read online and use critical thinking when you’re doing your own research” to every post.
The question is how much is it incorrect? Because the bot isn’t AI or anything. MBFC’s database is used in research and has been compared with other independent sources and deemed reputable enough.
Citation needed
our results suggest that there is substantial agreement across different sources of domain quality ratings, and that aggregated domain ratings provide a useful tool for advancing misinformation research.
https://academic.oup.com/pnasnexus/article/2/9/pgad286/7258994
Your source:
Domain ratings may not be as accurate as fact-checking individual pieces of content
You know – like a stupid bot writing useless bullshit.
but they offer a convenient tool for evaluating the efficacy of antimisinformation interventions
Also my source. You know, when used like a person with more than two brain cells would. Instead of nit-picking at the bot, why don’t we look at the bigger picture for the value it provides?
Honestly, the bias piece was never the important piece for us. It was the credibility piece.
Just trying to give some insight into why we used it in this community.
Then you understand the negative reactions. Especially regarding controversial topics such as Gaza where the bot preferred sources on one side to the conflict
Anyone who disagrees with me is a bot!
Love that they are actually using that reasoning…
Sockpuppetting? You have any indication of that?
We do. Admins found dozens of downvote alts and nuked them at the same time. Seems folks aren’t content to just state their opinion and leave it at that, and instead they feel compelled to overwhelm the system to give the illusion of uniformity.
These people were specifically trying to get the bot removed? Must have hit quite a nerve. I know it was biased in favor of Israel, but it must have been even worse. That bot sucks so bad people make mass sockpuppet accounts just to tell you they want it gone
You’re taking the wrong lesson from these findings.
I was joking. Just like you can’t be seriously claiming there is no consensus that the bot sucks and that all the net downvotes for that bot are due to a small minority of sockpuppeteers?
I didn’t claim there was no consensus, or that “all” the downvotes were sockpuppets. We have evidence that some of them were, which makes distilling the overall sentiment pretty difficult.
So based on your other comment, the “evidence” you’re referring to here is merely that vote manipulation had occurred in some other community?
Props to the LW admin who uncovered and banned the vote manipulation ring. Its existence is troubling.
I did a vote audit of the Soliciting Feedback from the Mods thread, and none of the sock accounts that were banned three days ago voted on the post or the most highly upvoted or downvoted comments. If you don’t believe me, I suggest asking an admin you trust to repeat the audit.
The outrage about the bot seems extremely organic, and any sockpuppetting going on is small compared to the overwhelming number of legitimate accounts casting votes that are apparent from the logs. The uniformity of the consensus does not appear to be artificial at all.
The news source of this post could not be identified. Please check the source yourself. Media Bias Fact Check | bot support
Go home, dude. You’re drunk. Oh. That’s me.
“Check yourself.” —The MBFC bot
So if it’s still showing up what does that mean?
the admin who manages the bot hasn’t had a chance to disable it for our community yet.
What if they say no to turning it off?
Just FYI, they agreed
Forgive me for having 0 faith in that admin
They said they would when they get to a computer.
But what if they never get to a computer??? 😬
Somebody find Sarah Connor! Skynet is behind this.
No one wants a bot.
I do.
Ditto
Why do you insist on fixing the bot instead of directing your energy elsewhere? Fixing the media bias bot to not have any bias is a fool’s errand.
The last sticky thread actually had some really good feedback, like using a fact checker that is part of the International Fact Checking Network (of which MBFC is not a member) and many other similarly great suggestions.
One of the issues might be in the name. We don’t want to create a bias bot. That seems like a fool’s errand, which is one thing we learned in the process of implementing the MBFC bot. We want to create something that makes people aware of posts that are from medium to low quality sources. Obviously, if the source is super sketchy, we’d delete it, but there’s a lot of grey area where we leave things up.
Mods should take note that this is how you listen to community feedback. Some actual learning is happening here, instead of doubling down we saw in the other thread
The other thread was an attempt to gauge feedback on specific ideas (as this post mentions, they are so in the works) and it precipitated this post
The other thread started from the assumption that the bot is useful and here to stay, even though the overwhelming feedback has been that it sucks and should be removed. It was a transparent attempt to increase support for it instead of an honest attempt at feedback. People still gave their feedback, of course, that the stupid bot should be put out to pasture.
At least now we’re seeing the bot is gone until improvements are made, the bias stuff is gone, the bot shouldn’t even appear except in select cases. That’s totally different than what they’re saying in the other thread.
We have been discussing the content of that feedback for about a week now, both with the broader moderator/admin community and within this team, and since most of us aren’t online at the same time (we have jobs) it takes a few days for the whole team to see and respond to opinions. Given that many of us disagreed on the best path forward, we had to come to a workable consensus. We have now acted on that feedback in accordance with the wishes of the community, so your claim that we had no intention to do so is significantly off the mark.
Maybe for you, but that’s not how I’m reading other people’s comments in there.
I’m telling you that the way you’re reading the comments has no bearing on what was actually happening behind the scenes.
Could do with out them banning some of the users with top comments though
There’s obviously no problem with incorporating other sources as well but, as I pointed out in that other thread, MBFC uses the IFCN for fact-checking per their methodology and Wikipedia page. They also explain why they use IFCN fact-checkers in their FAQ.
Good riddance.
No one wants this bot. In the last thread asking for feedback there was an overwhelming majority that did not want it. You’re attached to it because you made it. People don’t want it. It happens. Stop it.
I didn’t make the bot. I’m working on the aggregate tool, which isn’t developed yet.
The votes were overwhelmingly more positive than negative about it. We had the vocal minority against it, but the ones who wanted it let it be known.
Seems like we’re not reading the same thread https://lemmy.world/post/18775630
166 upvotes, 27 down. Same thread.
This is what you call “Not listening to criticism.”
Upvoting that post is not an indication of support for the bot.
Look at the comments in that post. Overwhelmingly negative feedback.
Sorry but you’re joining my blocked list. You don’t understand how any of this works if you think this way.
This is some extremely willful ignorance.
Nope, that’s not how it works.
There are instances that only allow up votes. There are people that will up vote any post by a dev as a show of appreciation for the effort, without necessarily thinking about or agreeing with the changes.
If you want a poll, then you have to do a proper poll. Up- and down votes are not it.
I want the bot gone but didn’t downvote the post. I support the mods reaching out for feedback, and also downvoting could decrease the visibility of the bot’s overwhelming negative reception.
Are you joking or something? Have you read any of the comments on that thread or this one? I really cant get how denial the mod team on some .world lemmy communities can be.
I read every comment. As I’m a mod here, I will only be civil, so I am keeping all my thoughts to myself. Have a good day.
And you have the balls to cite the upvotes on that thread as an argument when you know for a fact most feedback is against the concept of the bot itself? People upvoted that thread because the title made it seem the mod team was willing to do something.
…which we just did.
About half of the unique comments by my count are suggestions for improvements or expressions of support. The 10 posts with the most downvotes are all requests to remove the bot.
Let’s be realistic - this is far from consensus.
The 10 posts with the most downvotes are all requests to remove the bot.
These are some highlights from the top 12 posts sorted by “top”…
- “My personal view is to remove the bot.”
- “One problem I’ve noticed is that the bot doesn’t differentiate between news articles and opinion pieces.”
- "You don’t need every post to have a comment basically saying “this source is ok”
- “I think it should be removed”
- “My personal view is that the bot provides a net negative, and should be removed.” <- me
- “Partisan fact-checking sites are worse than no fact-checking at all.”
- “Remove it.”
- “MBFC’s ratings for “factual reporting” are a joke.”
- “This thread is a mess.” <- also me, sorry
- “The bot is basically a spammer saying “THIS ARTICLE SUCKS EVEN THOUGH I DIDN’T READ IT” on every damn post. If that was a normal user account you’d ban it.”
- “The majority of feedback has been negative. I can’t recall a single person arguing in its favor, but I can think of many, myself included, arguing against it.”
- “In literally every thread I’ve seen it post in, it gets downvoted to hell.”
To put it charitably, 2 and 6 are only mildly critical or express tepid support, at best. The remainder are… something less than supportive.
I understand that this is not a democracy, and that it’s ultimately up to your good selves to guide the community as you see fit. However, I think there are valid criticisms to be made regarding your collective ability to engage with feedback.
However, I think there are valid criticisms to be made regarding your collective ability to engage with feedback.
I don’t think that the mod team has ever said that there is not some valid criticism. Feedback from the community (not just the !News community) is precisely why we have made multiple changes to the way this functions, the layout, and inclusion of different sources.
There is a vocal minority of the community that feels the need to swear, engage in personal attacks, manipulate votes, accuse others of being paid actors, insist that “everyone” agrees with them, and so forth, which does tend to make it difficult to engage in a forthright discussion about what is best for the community.
I don’t think that the mod team has ever said that there is not some valid criticism.
That’s not what an inability to engage with feedback means though, it means genuinely listening to the perspectives of others even in cases where it’s inconvenient, unexpected, unpleasant, and yes in some cases presented in an offensive manner.
That doesn’t mean you need to just allow people to spew hatred and vitriol at you - of course you can call out that kind of behavior where you see it. It’s probably fair to say that some Lemmy users would struggle to express themselves on topics they feel strongly about without being offensive.
That said, I suspect that a lot of the vitriol you’ve encountered on this topic has in some part been provoked by the mods collective reluctance to actually acknowledge the many shortcomings of this bot or any potential reincarnation.
I mean the following in as congenial a manner as possible, but the comment of yours I replied to earlier (regarding the most downvoted comments in the other thread) seemed quite dismissive. You may not have intended it as such, nevertheless that’s how it appears. Engaging with that feedback would mean considering the actual content of those comments with a charitable attitude? JonsJava similarly quoting vote counts for the other post as a means to disregard concerns.
Again, in as congenial and constructive a manner as possible, Blackbeard has revealed that there has been some vote manipulation which I acknowledge has frustrated things from your perspective, but again the narrative that “there is a vocal minority opposing the bot and inflating comments in opposition to the bot” is dismissive of the very real issues extant.
I’m not alone in feeling exasperated by the cycle of request feedback > dismiss feedback.
None of the feedback has been dismissed. I don’t know how to say that any more clearly.
We read every comment. We have worked with other moderators of multiple communities to implement changes based on the feedback we received from the very beginning. This thread is just one of those examples. The mods and users are on the same side in this effort.
Please understand: the only people who are dismissing user feedback are other users when they say things like “no one wants this.” That’s literally dismissing the many users who express that they find it beneficial. We are working on developing community resources that will meet the needs of most of the people here. That process takes time because we are an asyncronous team of volunteers.
Edit: I should note that the “no one wants this” comment was not meant as a personal attack. I only intended it as a typical example of a reductive, unhelpful, and dismissive comment. I didn’t realize until after that you posted another comment on this thread that was substantially similar. My apologies for the inadvertent example.
First, admins have pointed out that dozens of accounts (now banned) were being used to artificially boost certain kinds of feedback and bury others, so if we’re not allowed to point to votes as a source of valid information, then sorting by “top” is equally invalid. Those could simply have been the comments those alts decided they wanted to push to the top, to make their point.
Second, we’re volunteers who have a few hours set aside each day to open a discussion into things that need to be updated or changed, and the vitriol that’s been hurled at us is disproportionate compared to the ostensible “damage” being done by a single automated script. One moderator threatened to resign over the hate that’s been blasted into their face. It took us less than two weeks to post a request for feedback, and then to act on that feedback. You (the disapprovers) all got exactly what you wanted. Pardon me for being blunt, but what the hell else are you expecting from us?
As stated elsewhere in the thread, my vote audit shows no participation from any of the 29 banned sock accounts the in the !news feedback thread, or this one for that matter. Please take the votes more seriously. If you’d like to spread FUD about the legitimacy of a vote, ask an admin to audit them first so you can state with evidence that a specific vote has been manipulated.
People trust the software to tell them what others are thinking, and if you successfully spread the false idea that votes that disagree with you are manipulated, you’re not just arguing in bad faith, you’re undermining the federated system we should all want to succeed.
-
Thank you. I don’t have that kind of audit authority and all we were told is that vote manipulation was occurring. We’d love to have you join the team if you’d like to help.
-
We took all of the feedback seriously because the bot is gone. I’m really not sure why people keep pretending like we haven’t already acted on it.
-
That you’d call this “spreading FUD” or"bad faith" is, frankly, insulting. I can only act on the information I have. In the end, I said that manipulation made assessing the situation difficult, but we still followed through accordingly. We are volunteering our time, and you lying about our intentions isn’t helping either.
I only have a few hours per day to devote to this. If you think you can do better, then step up.
-
We want the bot gone. That’s it. It’s really that simple.
I’m sorry if I come across as preachy in the below post, but I wanted to try and explain to you where the critique is coming from. And also that it’s not personal or any widespread resentment.
I (and many others) get what a thankless and also necessary job moderating is. It’s not easy to do it well, it’s frustrating, it’s thankless and without it the community would be dead. But being a moderator and sticking out your neck brings you exposure and you are guaranteed to meet more asshats than you ever thought existed. But the users are not one homogenous group, it’s not because one user has flung abuse at moderators, that all users are now suddenly resentful of moderators.
The person you are replying to, put a good bit of time in listing what comments were most up voted, which are probably the comments that found most support amongst the users in that thread. In the same way that we should not be dismissive of what you do or say, you shouldn’t be dismissive of what others do or say (or up vote). Mutual respect and all that.
Self reflection is also important, it’s important to realize and accept that it is possible to be wrong about something. Doing a mea culpa and moving on is far easier in the long term than doubling down and digging a deeper hole, yet it’s a lot rarer because it hurts our ego in the short term.
Their final point about a problem with handling feedback rings true to me:
-
You (not you personally, but the team that did that feedback thread) have apparently treated up- and down votes on a thread as a poll and a popular mandate for action, but up- and down votes are not a poll and most (probably most) people don’t use them as such.
-
Up- and down votes on comments are useful for finding which remarks resonated with or turned away other users. They are not a poll either, and most upvoted are not automatically most correct at all, but they give you a chance to read the room.
-
You (now you personally) have thrown shade on the people that up voted comments against the bot, by insinuating that those people might have been bots themselves and that therefore their opinions are irrelevant. Yes it’s possible that there are some users using alts, but all those users? Not very likely.
-
The best feedback I saw in that thread was not in the up or down votes, it was in the comments themselves. There were some very compelling arguments as to why using a biased site to display bias, was a bad idea. Those comments also had quite a bit of upvotes, so the way I read the room, that was a popular sentiment.
-
The person you are replying to made a few arguments and one scathing critique which they probably hoped that you would improve on in the future. Imo a polite disagreement with your previous statements. You respond by being dismissive of his arguments and acting like it’s a personal attack. They were sticking to facts, you’re making it about you as a person. I really don’t think that was their intent.
Yes, it is personal. The fact that one mod almost resigned over this means that whether or not people intended for it to be, the criticism was in some cases very personal. And we have evidence that some (never said “all”) voting was manipulated, which makes our job more difficult because there’s no way to tell how many of those comments were upvoted because many people agreed with them, versus a few people agreeing with them so intensely that they were willing to break the rules to prove their point. At the end of the day, people gave feedback, we reviewed it, and the bot is gone. We didn’t ask for it to be created, had no role in coding it, didn’t ask for it to be rolled out, didn’t turn it on, couldn’t change it, couldn’t turn it off, and gave the admins time to try their experiment while we determined whether or not it made sense for the community. It wasn’t our bot.
-
Thank you mods.
Don’t grade sources on leftness or rightness. It’s relative. Aggregation doesn’t fix the problem of Overton shift.
My goal with the composite score is to grade based on how factual they are, not political leaning.
