I find humour in the “clear peer review” part. I’ll take that with a grain of salt.
Obligatory Stewart Lee clip: https://www.youtube.com/watch?v=FzOv14fA-BI
To be fair, journal articles and scientific research in general have gotten to be pretty bullshit. Haven’t they studied this and proven the vast majority of published journal papers probably shouldn’t have been?
A couple easily Google examples of discussion regarding scientific publications likely being bullshit.
Why Most Published Research Findings Are False
Surge in number of ‘extremely productive’ authors concerns scientists
Too much academic research is being published
More than 10,000 research papers were retracted in 2023 — a new record
Whistleblowers flagged 300 scientific papers for retraction. Many journals ghosted them
Top 10 most highly cited retracted papers
And on and on. Publish or perish and general shitty culture in academia is why I quit phd and took my masters and left.
I saw a clip on how kids out of uni don’t believe anything not peer reviewed; even intuitive observations in nature that otherwise undocumented or site specific observations that went against the grain.
Science is a way of thinking and observing, rather than papers, but papers are a good way to refine your thinking
TBF I’ve lost count of the number of times someone has cited some paper as a reference for the point they are trying to make and when I inspect the paper it has shitty “n”, the paper is written for an agenda (not sure what that’s called where I.e. a paper saying smoking is good for you/not harmful is paid for by the tobacco industry and written by tobacco industry scientists), or it might even just be straight up bullshit written to look like a legit paper.
Peer Review at least offers some barriers to the problems with papers, but it’s definitely not a panacea.
In theory, a paper gives you a methodology that you can use to reproduce the findings. And a refusal to use papers to repeat findings (because shit costs money and nobody wants to publish iterative studies) means you end up with a bunch of novel findings that are never confirmed through repetition.
But the fact that nobody is bothering to repeat these studies also raises a question of what exactly is being researched. Certainly, the more useful scientific research efforts are about formulating applicable techniques. So they would need to be reproducible to have any functional value.
The fact that we’re not seeking to replicate studies suggests that we’re investing a ton of time in niche under-utilized fields. And that may be a problem of investigative research (we’re so focused on publishing that we don’t care what we’re actually studying) or a problem of applied sciences (we’re so focused on scaling up older methods to industrial scale that we’re leaving better methods of production on the cutting room floor).
But its definitely some kind of problem.
The problem is without peer reviewed papers it’s hard to credit that someone all the way around the world observed something.
In a perfect world nobody is lying and everyone has the scientific base education to understand how to report phenomena properly. But uhhh… Yeah.
I’m guessing not all hypotheses receive the same interest or funding to begin with. Definitely seems to be a selection bias on what actually gets funded/studied. Even worse, when they withhold results they don’t like from being published.
Solution: don’t give a fck on the brain farts of random people.
Lol had the same experiance with a guy called stamets . He didn’t believe i was a psychologist because “i wrote like a teenager” like english isn’t my first language bruh i am tring here alright just cut me some slack . And then the mod went and deleted my post because i was “harrasing” people to be fair to the mod i was but so was they like i bet the stamet kid could never speak my first language like i speak english .
Okay but setting aside the details of the truth about language, having a degree should not give you or anyone carte blanche (definition: “complete freedom to act as one wishes or thinks best”) to harass strangers on the internet. I know nothing of the incident you referred to, only what you said here. Also btw I did not down-vote you.
Regardless of whether a mod is a literal child or not, that is the “role” that they have stepped up to fulfill - to be a curator of whatever community, or instance, or whatever - and should that not deserve at least a modicum of respect? i.e., if you stepped up to fulfill that role for your own community, wouldn’t you want people to respect you in turn? From your words, you obviously do, so why not offer it preemptively?
Especially as a psychologist: you better than most people know that you get what you give, especially when dealing with children.
They likely were saying that the truth of whether you were a psychologist or not was irrelevant, what matters was you breaking the rules of that community - b/c at some point, if truth is functionally indistinguishable from a lie, then does it matter, practically speaking?
Anyway, I hope you enjoyed this food for thought:-).
Hmm i have thought about it and asked tge mod to give stamet a similar warning too but he just ghosted me
Cool, so what’s your point? Why would they assume you’re anything but another “guy on the Internet”? For what it’s worth, i have a hard time believing you’re a medical professional too
My point being this is what the meme being about 🤦 and frankly i don’t care what you think i shared my experience because this is lemmy and that is what it is for.
Good for you. Get yourself a cookie
And next on Fox News, we will hear from the experts both sides of the issue, the researchers and the internet jackass.
“the jackass researchers and the internet expert”
Well, sometimes there’s another step missing just before the Bullshit: “Use the small, narrow findings to inform a greater narrative beyond the data’s scope”
I love how all the comments in this thread are like “yeah but it is bullshit tho!”
Well I’d like to think I’m not! I wanted to point to an actually dubious thing where we might call into question a study, so we could still respect the work being done while validating the importance of keeping standards in research.
You’re right though that it’s disappointing how many responses seem to address only the flaws in modern science and not acknowledge the strength of the scientific process. I think a big part of it does come down to how scientific findings are interpreted and reported to the public, and even further an all-too-human misunderstanding of epistemic limitations. Our cultures should spend more time educating people about the limits of knowledge and fact, how they are constrained by other flawed systems, etc. That would be a half-decent start, if we could only fix the entire reporting problem too.
Thank you for pointing this out.
Yeap, that sounds like my reviewers :(
Hasn’t read the article methods but still decided to comment: cOrReLaTiOn dOeSn’t eQuAl cAuSaTiOn
Yeah but also just publishing correlation is a shitty practice. That’s supposed to be a hint to look deeper, not the end conclusion.
There’s a generation of internet debate guys who seem convinced that correlation disproves causation
Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there’.
herp doesn’t imply derp
All my literal this
this is the way doggo puppers, hecking upboats to the left
guy on lemmy “this was already obvious, why don’t they try studying something actually useful”
This, but to some degree, unironically. If studies aren’t reproducible (or deemed worthy of reproduction) then there’s definitely a disconnect between the folks handing out research assignments and the folks engineering applicable solutions to scientific problems.
That goes two ways. You could be a guy who successfully formulates a mathematical model to support the existence of Neutrinos and face a funding board that has no interest in building a LHC. That’s arguably a problem of malinvestment within the scientific community. Or you could be a guy who successfully formulates a mathematical model for a new kind of mouse trap that’s 10% less efficient than traditional mouse traps. That’s more of a university research assignment problem. Or you could have a researcher who claims he’s the only one who can do a particular thing, because he’s got the magic touch. If the research is unfalsifiable by design, that’s an entirely new kind of problem.
i think you bring up valid instances where this is fair.
but i think i’m speaking to the very obvious and important ones that are worthy of reproduction. like i’ve seen articles be like “these corporations are responsible for 99% of climate change” or something
and the comments will be like “duh we knew that”
which true, but not empirically. being able to cite data from actual research from professionals is so valuable and far better than anecdotes or guesses.
that said, is there some way for a layperson like me to identify when research is not deemed worthy of reproduction? or is it a lost cause
which true, but not empirically. being able to cite data from actual research from professionals is so valuable and far better than anecdotes or guesses.
While its certainly helpful to get the raw numbers down on paper, you don’t need a filing cabinet full of documents to recognize that fossil fuel consuming electricity producers and airliners and manufacturing centers the but-for cause of climate change. Fossil Fuel goes in. Carbon emissions come out.
We can definitely use a more meticulous bit of R&D to find exactly where and when these emissions peak, in order to reduce total emissions without sacrificing an abundance of economic productivity. But “did you know burning the fuel makes the pollution?” isn’t a shocking conclusion.
Where things get annoying (and where in-depth research genuinely comes in handy) is in the functional policy that follows this recognition. Once you know a widget factory in China is 10x less efficient than its counterpart in the US, you can formulate a trade law to limit imports contingent on reform. But as soon as you start impacting some retailer’s bottom line, you get some screamer ad “Congressman Greenpeace Wants To Make Your Widgets 10x as Expensive to Save The Stupid Spotted Owl! In Truth it is the Spotted Owl that produces all the emissions! Kill the Spotted Owl!” financed by the worst people you know.
And that’s when you get some facebook troll group (or marketing team or bot army) spamming “Spotted Owl Farts Killed The Environment While Joe Brandon Clapped!!!” And then it becomes orthodoxy in the denialist community such that you’ve got Sunday Morning talk shows with people arguing over Spotted Owl emissions rather than trade law.
is there some way for a layperson like me to identify when research is not deemed worthy of reproduction?
Not practically, no. As soon as you’ve got that kind of info, you’re no longer a lay person.
At some level, you need a network of trust with someone who does know and does have a serious take on this. And that network is going to be informed by who you already trust and listen to. And that’s going to be informed by who they trust and listen to.
That’s the real terror of the modern mass media system. We’ve corrupted so much of our information stream that its genuinely hard to find a serious media venue that’s not been gobbled up by a for-profit marketing firm.
If after all that preparation, your pride can be pierced and wounded by one of myriad neckbeards or Karens on twatter, you might need let go a little bit.
Maybe that guy was just one of the people who worked on one of the 19 other studies that didn’t publish because of the negative result
Alternatively: Be pressured to churn out papers by the university’s MBA-crazed leadership, make weakly-supported assertions in order to make a paper exciting enough to be published. Your peers in academia and industry call you out on social media when they become aware of your dubious claims.
…obviously, that’s an extreme situation. It’s true, usually the people working with a given subject on a daily basis will have a better grasp than random, disreputable voices on the internet. Being critical of sources and reasoning is important.
Doesn’t even take direct pressure from others. Getting published is one of the best ways to gain access to funds/resources, and just like with every other profession many will succumb to the temptation to take shortcuts or fudge the truth in the pursuit of money and/or prestige. I knew one woman who gave up on pursuing a career in cultural anthropology because she had come to believe that getting published was more of an exercise in creative writing than in actual science.
It’s actually much more common than people think. Oh your numbers don’t match what the rest of every else’s says? Fudge the numbers a tiny bit nobody will notice. That way when you have to defend your work it’s a little easier because it’s in like with other work.
The worst part is when that guy’s right.
Yup. They forgot that sometimes what’s actually happening in that one line is-
- Go to School for a Bachelor’s Degree
- Get 10 years working experience in specific field
- Watch researcher whose never stepped outside of a lab make assertion counter to real life.
- Call Shenanigans
- Watch the findings go nowhere
Dunning Kruger curve. The people who know the least about a topic speak the most confidently about it.
Don’t think it’s exactly Dunning Kruger. We all think about the curve of gathered knowledge and perceived knowledge.
But they didn’t even start to gather knowledge, they just respond with something that sounds truthful and fits their world view in order to feel better without doing anything.
But hey maybe that’s just my Dunning Kruger talking.
I see this name everywhere these days. I think… I’m having a Baader-Meinhof about Dunning-Kruger
Nice we’re keeping the Reddit tradition of just repeating “Dunning Kruger” every time we see disagreement
