Paco Alvarez: Why is fact checking important? How does the type of fact checking process differ from other publications?
Maha Ahmed: Okay, I can answer that question. I think fact checking is important because it protects the organization, the reporter and everyone involved with the story from legal action, which is every journalist ‘s worst nightmare, I feel, have the retribution of, like a difficult source or an antagonistic source.
But also from an ethical standpoint, it’s really important to just make sure that what we’re putting out into the world is really vetted and verified and for readers when they’re reading our stories to know that what they’re looking at and what the information that they’re taking in is confirmed as accurate. I think one of the most underrated benefits to fact checking is that reader-trust element.
I think specifically at Type, from what I know of other newsroom processes, I think that it varies, but we tend to be a little bit more on the rigorous end of the spectrum, I would say, which I take a lot of pride in. And I hope that our fact checkers also do, who’ve come through our research program and have worked at Type over the years. But I would say that really from like what I’ve heard from various reporters, it’s just like kind of a pain in the ass to be fact checked at Type – in a good way for the most part! From what I’ve heard and really everyone’s kind of getting just like in really intense scrutiny on basically every single sentence in their story. I think our fact checkers are always catching things that reporters didn’t even think to think about in the reporting process, which makes sense, kind of highlights the importance of having that mechanism in place so that there are other people thinking about the questions that reporters are kind of mulling over for months and months and months.
Nina Zweig: One of the benefits of fact checking that I think is less frequently discussed, too, is the fact that when you’re doing a very rigorous fact checking process, oftentimes the checker will uncover new information to supplement what the reporter has put into the story during the fact checking process. So sometimes we’re even building upon people’s existing reporting and helping them make their stories better that way.
Emma Davis: I guess just jumping off of Maha’s point about fact checkers learning as they cycle through the program. I’m currently doing the yearlong program, and I can say that in terms of my past experience, too, Type’s process is just so much more intensive in terms of really being like a word for word approach versus like just checking general overarching facts in the piece or names or things that people would commonly think of, it really is more of an intensive process than I had previously had coming into this experience as well.
Ahmed: And another kind of underrated element of, like, why I think the fact checking, specifically that we do at Type is so key is that, it’s sort of like a mirror of the reporting process in general, which within the accountability framework that we operate in, is ideally taking these institutions and these people who have a lot of power and scrutinizing the claims that they’ve made or the actions that they have. And I think the work that we do as fact checkers where we’re deciding how to evaluate a specific source and like how much credibility to lend it, whether it’s coming from like a police department or like some other government source where maybe it’s an official narrative of some kind, but we want to make sure that that’s corroborated. I feel like it’s a microcosm of the kinds of decisions that reporters and journalists in general are just making on a day to day basis. And like that same that same sort of accountability framework applies within the process of fact checking itself.
Alvarez: Something I think all three of you kind of touched on. But how does the Fact checker work with the reporter and editor on the Story? Where do they fit in?
Zweig: So where the fact checker fits in the process depends on the individual story. But one of the ways in which our fact checking process at Type is unique is that the people who are doing it are often the same people who are helping reporters with research on their stories earlier in the reporting process. So sometimes they will even be integrated to a certain extent, while the reporter is still working on the story and then possibly the same person will even come back later to work on fact checking that story.
But in terms of the general process, the way we do it at Type is once a story has been edited, the fact checker will meet with the editor and discuss any potential difficulties that the editor thinks will come up during the check, any facts that were particularly tricky or material that’s still outstanding. And then the fact checker connects with the reporter, usually through the editor. And at that point, the actual fact checking process begins. if anyone wants to discuss that.
Ahmed: One thing I’ll say is that I feel pretty proud of how we work at Type in terms of the fact checking process is that, I think the fact checker holds a lot of weight in terms of their work and their suggestions and feedback to the editor and the reporter in the process. So I think I have you know, I’ll all of us have fact-checked in like a few different newsrooms before and in my experience, it sort of varies depending on the way the newsroom is structured. But sometimes fact checkers, you’re not even really interfacing directly with reporters. Sometimes their changes are more suggestions than taken as potential actual edits. And I think everyone would agree that fact checkers are seen as equal partners in that process, which I think is really, really important in order for the fact check, as a process to work as well as it can. And I think that’s when our best work happens.
And in terms of the actual process – I’m not sure how specific to get, but what I’ll say is that usually when the fact-check is starting after the editor’s, and myself as the head of the research team at this point in time, kind of get everyone on board on to the process, basically, the fact checker is working closely with the reporter to go through all of the sources that they have handed over for their story. And just looking back through that sourcing and making sure that not only is the stuff that’s cited from that source accurate within the piece itself, but also just making sure that any kind of interpretation that’s going on, any sort of context that’s being given, isn’t missing or isn’t somewhat misleading. I think that’s a really important part of the fact check process in addition to just verifying smaller details.
And then, in keeping with that spirit of like, you know, equal partnership that I mentioned before, the fact checker meets with the editor at the end of the process once they have gone through and looked at all of that sourcing to give their feedback and edits and talk through any bigger questions.
Davis: And I guess to just add to the process question, I think something really unique to type as well as just the time that we’re afforded to do fact checks for these types of intensive investigations. They can run for weeks and you actually have time to dig into really every word and every detail. And I think that also lends itself to having a really interesting relationship, balancing relationships between both the reporter and the editor and fact checker and how that all works, because it’s as much, as Maha always says, about diplomacy as it is actually getting into the details of the actual work itself. So that lends itself to a really interesting process. I think, too, just as someone who’s newer to fact checking and Types process as well, I think the fact that stories can and do change so much from the beginning of the fact check, to when they officially run, does speak a lot to how much power the fact-check processes has and just how important the process is, as the stories been can change dramatically.
Ahmed: Yeah, honestly, like Emma mentioned, I will say like fact checking probably more than some other parts of the journalistic process really does test your interpersonal skills, I’ll put it that way, in terms of just you’re really making sure that you’re not only being as rigorous as you can in your work, but also making sure you’re maintaining that relationship with the reporter and the editor and the reporter’s sources and framing your feedback in specific ways where you’re doing a lot of reassuring in the work that you do, which it’s a vulnerable thing to like work for over a year in a lot of cases for the stories that we do on something that, maybe a reporter has been like spending all of their time on. And they just get so deep in the process and they can be quite vulnerable and doubtful by the time they get to our stage. So in addition to like making sure everything stands up to that scrutiny, it’s also kind of like serving this reassuring role that your story is important and there’s really valuable stuff in here.
Alvarez: A lot of Type’s investigations deal with information from anonymous sources or leaked documents or contain allegations against specific organizations or government institutions. What are your approaches when fact checking legally sensitive information?
Ahmed: In terms of dealing with legally sensitive sources, I think you know that material can be kind of challenging because it’s both the most important material to get right and spend a lot of time on and also can be the trickiest to verify depending on like what the sourcing is for that material. But we luckily do have the help of a lawyer that advises Type on different investigations to sort of help make those calls, and making sure that we’re not saying anything that could necessarily get us into trouble.
I think the answer to that question really is that it is so important to make sure that the claims that you’ve identified as legally sensitive are super airtight. And I think a good rule of thumb is that those are the claims that need to have as much corroboration as possible. So like, let’s say for smaller details, you are only looking at like a couple of sources or even like one source to just back up a claim. I think with legally sensitive material, the more sources you have that kind of speak to that claim and its accuracy, the better because you kind of have to do this sort of like shift in mindset where we’re thinking about what could be putting us in a difficult position if this were to make its way to the court. You know, did we do our due diligence? And like the way to show that is that you’re not just taking the word of like one source who, you know, shared a piece of information with you and are really like going through exhausting all possible avenues to verify that information.
Zweig: So I think one particular type of legally sensitive fact in a story could be I’m just going to give a very vague example: An evaluation of someone’s qualifications or credentials in their field, for example. And as the fact checker, when you’re in that situation, as Maha pointed out, you want to think about how theoretically down the line you would be able to show someone in a legal setting or otherwise that you had done everything you could to verify that that claim was accurate and fairly framed.
So if you are looking at a fact that is, as I said, evaluating someone’s credentials or qualifications, you as the fact checker would want to go to as many experts, for example, in that person’s field as possible and interview those people and ask them to evaluate the claim that the reporter is making in the piece. And often that work is kind of invisible in a piece because you’ll quote maybe one or two sources evaluating a claim but underlying that, there should be an additional layer of many more sources, the more sensitive the claim is.
Davis: Yeah. I think Nina’s underlining a really important point of just our process, too, in general in terms of fact-checking is that it’s not just about finding sourcing through online databases or things that are already out there and that you can research online. But reaching out to attorneys or experts or other people in the field to really get a sense of the accuracy of a quote. And that’s part of the entirety of the fact checking process, you’re reaching out to actual human sources, both included in the piece and not.
And I think this is just a general framework for all facts, but especially sensitive ones. One of my favorite things to do is to try to find everything that I can to disprove a claim versus just trying to find a lot of corroboration. Because with so much information out there, it can be kind of an endless pull of verifying the sources that you are finding on about a given topic. So approaching fact checking in that sort of way, too, is always helpful.
Alvarez: What are the most difficult checks you all have done? What made them difficult?
Davis: I’d say some of the most difficult checks that I’ve done is I’ve checked the two stories that we’ve put out so far for the Inside/Out Journalism Project, and that was a project Nina spearheaded to support incarcerated reporters. And fact checking those stories have, of course, some more unique challenges because the reporters themselves are incarcerated, which really limits the communication that we can have. And you have to kind of approach it in a different way, just given the different logistics of what that looks like.
So for the latest piece that we published, it was an analysis piece from Rahsaan Thomas. And while a lot of the sourcing documents and notes and text of the article itself can be emailed to us at the office during a more time sensitive fact check procedure, mailing isn’t feasible because it would be delayed by several weeks. So we really had to solely rely on, of course, phone calls. But given that Rahsaan was incarcerated at the time, that was very unpredictable, was only given brief amounts of time to talk to us on the phone and they were often not able to to be scheduled. So the editor and I were kind of on call for whenever he had time and we could ask him questions. So that really limits the just back and forth we were able to have with the reporter because in a typical fac-tcheck we’d be able to email or follow up with calls as needed. So we really kind of had to tighten up the process and communicate in a different way.
And I guess navigating the back and forth of all of those calls and a lot of the actual like review of fact check changes was different because we’d have to read word for word the changes to Rahsaan over the phone, which is different than having a reporter who has time to sit down with the text and comment on Google Docs or go back and forth about changes and approval. So I think that was the most typical type of check that I’ve done just in terms of the logistics of it. But I think the fact that we’re able to still do a fact check of that nature really speaks to the uniqueness and the just efficiency of our process, that it can be adapted to those types of situations. It just looks a little different.
Zweig: So one fact-check that in the past few years that I’ve been doing this has really stayed on my mind is actually the first feature story that I ever fact checked at Type, which went through the fact check process in March 2020 in the middle of the month and was about 10,000 words long.
It was a story on the case of Nikki Addimando, who was a young mom living in upstate New York with two kids who was prosecuted for the killing of her boyfriend, her kids father in what she said was self-defense. And the story was about the larger trend of survivors of sexual assault and domestic abuse being criminalized for defending themselves in situations in which their lives were at risk. And the reporter of that story, Justine van der Leun, was wonderful and had spent – off the top of my head, I can’t remember how long, but a very long time working on it and was very devoted to it.
But just the length of the story and particularly the sensitivity of the material, made it very, very difficult. And that, I think more than any other story I’ve ever worked on, required just such an integration of the checker into the world of this story. Like really absorbing Nikki’s entire backstory and history, the history of her relationship with her partner and how their lives had been in the few years before his death. And just at the same time, the story also involved a lot of interviews with people who were close to the case in personal ways and discussing a lot of very traumatic material with them. So I really had to try to find a way to strike a balance between having those discussions with people in a way that didn’t feel too kind of clinical or too journalistic, because obviously when you’re asking someone to recount something very traumatic, you don’t necessarily want to come across as if you’re just poking at them for information. But you also have to be careful about the way you speak to them, because in the end, you are having that conversation as a journalist and specifically for fact checking purposes in this case.
But yeah Justine ended up building on her reporting on Nikki’s case in subsequent stories. And as part of the attention that her work brought to that case, Nikki eventually retained counsel and went through the legal process again, and her sentence ended up being reduced. And yeah, just a lot of national attention was drawn to her case and to the issue of criminalized survivors in general. So yeah, I think that’s an example of a very difficult, very legally sensitive story that had to really go through the ringer in the fact checking process to make sure that everything was airtight. And in exchange for that, you got a payoff at the end in terms of getting the story out there and getting this source’s story heard.
Ahmed: It’s a little less juicy because it’s not specifics that I’m getting into, but in broad strokes, the kinds of stories and the kinds of sources that I feel like really in some ways test the craft of fact-checking the most are ones that deal with specifically sensitive sources. And what I mean by that is, specifically sources that are in extremely vulnerable positions and coming from maybe vulnerable backgrounds, and it’s the riskiest for those people to be even participating in the journalistic process, but then doing it again in the fact-checking process. Nina’s situation that she described made me think about this.
And I think the hardest thing is making sure that you’re not putting the source that the journalist has already spent all of this time developing these relationships with in any kind of uncomfortable position, any kind of like state of agitation or anything like that. And I think there can be some pressure when you’re a fact checker on a story like that to make sure you’re not putting the source at even further risk when the story comes out by getting everything correct. Because I always tell fact checkers when I’m training them, which I do as part of my role at Type – and that I tell reporters when I’ve done the like fact checking presentation for some of our reporting fellows– is that you’re really doing your sources the biggest service by going through the fact checking process in the end because it’s their experience and their story that’s going to be picked apart and potentially torn to shreds if there’s like one error in it when it comes out. And that can do a lot in the way of their kind of ability. Especially when you have sources that are at risk of legal action against them or at risk of detention or something that risks their citizenship status – like anything like that that is really tricky in terms of consequences, I feel like it’s sort of where I find the fact checking process to be the most difficult and often like the most rewarding if it goes well. So I think that’s just one thing that I’ll say. But it’s like it’s very stressful at the same time.
Alvarez: My last question to make things to make the fact checking process less stressful. What are things you wish reporters and editors did to to make the fact checking process easier?
Ahmed: I can talk a little bit about this first. I think about this a lot because I would say as a person, like on a day to day, I’m not the most organized person. And yet I feel like in my role as previously a fact checker now kind of running the fact checking assignments at Type and getting everyone on board to the process, like the biggest thing that I’m always kind of emphasizing to people and also that makes on a logistical level, fact-checking super hard a lot of times, is when people aren’t organized. And I feel like a freak because I’m like always just getting people to be as meticulous as possible, which is kind of – it’s not super intuitive a lot of times that the way that you organize your material and even the way that you refer to it in your annotations when you’re getting ready for the fact check, would have any consequences for how well your story is fact-checked, but like I would say, that’s like the number one thing that causes problems is just like lack of organization, of sources or material, in my experience.
Zweig: One thing that I like evangelizing about mostly to other researchers and fact checkers is the utility of annotating your draft as you are writing it, as opposed to before the fact checking process is going to start. So I forget if we mentioned this specifically yet, but the way that we have reporters provide us with their sourcing before a fact check is by footnoting facts in their draft. And I have always thought that it would be a good idea to integrate that earlier into the process and maybe try to do something like that while you’re drafting.
I’m actually working on a draft right now and I’ve been doing it with endnotes so that I don’t have a giant chunk of footnotes at the end of each page, and it makes it more manageable to read the document. I understand why reporters do not want to do this, but it ends up being very helpful in the fact-checking process. And I do think it’s very helpful for organizing your reporting as well. So, yeah, that’s my cause.
Davis: Yeah, I guess just to really add to that very briefly, I guess fact checking at Type isn’t an afterthought so I guess I’d say that it’s important that reporters also don’t view it that way. And think about it as Nina said from the start of their reporting and keeping track of sourcing and how they’re getting to conclusions and all of that from the very start of the process in general.
Ahmed: And you know, I’ll add to that, sometimes I feel like the like level of organization and just being on top of your material, that is kind of required for a successful fact-checking process is somewhat antithetical to the the archetype of the bootstrapping, roving reporter who’s just on the ground and scribbles their notes on like a notepad and like is getting people to confess all this material. And I think that that is sort of like romanticized version of reporting that often doesn’t lead to a lot of protection and honestly, a lot of accuracy in a final product unless you’re taking that and translating it into something that can be checked.
And I will say like, you know, kind of going along with that in addition to ongoing organization, I think attitude is really important. And I think that a lot of reporters just don’t really like being fact-checked. A lot of them do, and those are my favorite ones to work with. And the ones who can see the fact-checking process as sort of a challenge to their credibility or questioning their skills and –some reporters get quite argumentative with fact checkers who are kind of just doing their job and asking questions. And really, at the end of the day, what I think a lot of reporters could benefit from keeping in mind is that fact-checkers aren’t in most cases getting credit for their work. And so the only reason that they’re really spending all of this time doing that work and asking you those questions and like doing things that maybe feel like interrogation sometimes are to support you as a reporter and make sure that you’re not in any trouble once the story comes out.
And so I think the more that reporters can see fact checkers when they do have fact checkers working on their material as, again, equal partners in the process and really there to support the story because ultimately that’s what’s most important in any editorial process is like the integrity of the story itself and the people that it’s about.