Lucas and Paul talk about how their research group utilizes peer feedback on all levels of research: from clarifying research ideas to polishing finished manuscripts. They depict how regular peer feedback allows them to publish better papers faster, and in general progress faster with their research. Watch the interview or read the transcript to get useful tips for your own peer-feedback practice.
Lucas and Paul are members of Dr. Viola Priesemann’s research group focused on “Physics, Complex Systems & Neural Networks”. They are a lively group eager to interact and cooperate.
I have been invited to work with this group several times over the last two years. During this work I have developed and tested my method for effective group feedback.
In spring 2020 they published a Covid research paper in the prestigious journal Science. The paper has been submitted at the beginning of April, just few weeks since the beginning of the pandemics. So the research and writing has been done really quickly — but I was pleased to see that the quality of writing didn’t suffer from this tight timeline. The authors have done a great job and I was curious to hear how was their process and whether peer feedback helped them with this ambitious project.
Now you can learn how this group is improving their work and writing by regularly feedbacking everything from project ideas to manuscripts. Lucas and Paul are sharing their experiences — successes as well as struggles — and offer advice to everyone who would like to get started with peer feedback.
Watch the interview or read the transcript below:
How training in peer feedback is useful
Martina:
Hello Lucas, hello Paul, I am very happy to have you here today. … We are here to talk about peer feedback, the value it can provide and all the stuff around scientific writing that goes into that.
At the beginning when I came to your group, peer feedback turned out to be the big thing for you guys. I mean, of course you learned the other basics like paragraphs and sentences and some productivity stuff but the peer feedback I had the feeling was something special. Can you comment on that?
Paul:
I think mostly it’s because before we have not really reflected on the feedback. It’s important that you get to know the rules of feedback once and find out how how to deliver feedback effectively. That’s something that is easy to learn and super useful but you have to be aware that’s something that you have to practice.
Lucas:
Yeah, I think that’s really the point, the practice part. You learn the concepts during the workshop but it’s not like “okay now I learned it, now I can apply it”. You have to beat that in into your head so to say [laughs].
Effective peer feedback needs practice.
Paul:
I suck at this, I suck at this so badly. It’s like the other day I was explaining something to a friend. It was some R I didn’t really understand it myself but she needed help and I was trying to explain and give feedback on her code and I was thinking to myself “I’m doing this all wrong”.
Lucas:
I think that the peer feedback groups help to remind you of all the things that you wanted to do better and you keep iterating on these steps and improving. Otherwise we would just go back to business as we did before.
Rules of effective peer feedback
Martina:
So what were the rules of good feedback that you learned?
If you just trash talk someone of course your feedback will not be heard. People only listen when they somehow like what they’re hearing.
Paul:
One big aspect is “how is the feedback perceived” concept. This includes what do you focus on but mostly how do you communicate it. So if you just trash talk someone of course your feedback will not be heard and then the whole exercise becomes pointless. That’s something where we still spend a lot of time with playing around and experimenting. I mean you noticed, Martina, our feedback culture is rough and very often there is no fluffy flowery words that sugar coat things if they don’t deserve to be sugar coated and that’s tough, right? You have to know what people you can do this with, So try to be nice is probably a good summary and you can dial back on the artificial niceness if people know you and have training. But that’s a very important step: people only listen when they somehow like what they’re hearing.
Lucas:
Yeah, that’s from the side of people that give feedback. For the feedback seekers it’s really important to be specific. It doesn’t help to say: “okay here’s my text, now give me feedback” because then people will get into these small details where you’re also upset that they now want to edit those parts and…
Paul:
… and they can’t help you with it, right? It’s because everyone focuses on something specific that does not improve the document overall. Because then you have another five construction sites in addition to the 10 you are already working on. [..]
Feedbacking as a part of everyday work
Paul:
The whole feedback thing became more of a subtle part that’s a constant background condition. Across various projects you always write on texts and you always collaborate with people, so there is feedback happening all the time. And it’s good to to stick to these guidelines whenever they’re required so I can’t really I can’t pinpoint how often we have feedback rounds anymore. To me it feels something very natural now, something that is constantly there.
Martina:
But if I understood it correctly, then it’s mostly among co-authors, right? So if you are collaborating with someone, then this is the person who will give you feedback. Or do you also ask for feedback other group members that do not collaborators on the particular project?
I can’t pinpoint how often we have feedback rounds anymore. To me it feels something very natural now, something that is constantly there.
Lucas:
Both. I wrote a paper together with Fabian and we had of course iterations between ourselves. But also at some point we put it out and we got feedback from people that were not at all involved in that research. And that was really, I think, where the decisive feedback came from. Where we said: “okay, this communication doesn’t work at all”, “this point is not clear” and so on.
You also mentioned the point what to to specify if you don’t know what’s the problem. Typically it’s the easiest thing if you think there’s a paragraph where the message is not clear and just ask a kid “do you understand what this is about”? And then people will immediately see “okay I have no idea what this paragraph wants to tell me” Then you can together look for the problem where the communication failed. So it’s hierarchical, starting with the most fundamental problem which is: “can you understand what’s in this thing?” and then you can go down and see if it’s the sentence structure or… and so on.
Martina:
Yeah, nice! Paul, you were busy with the Covid paper, I guess, until recently?
Paul:
There was another one, driven by Jonas and Seba, and this was for me a nice situation to give feedback to an existing project. I did not have anything to do with that because I was on vacation. Then I came back and Viola wanted me to do figures and to look over the whole thing. I came completely from the outside and could just say: “guys, what is this?” And that is refreshing for both parties, for both sides I would say. Because you get something external that has this blank page and if notation is not clear you will find that out very quickly. You’ll find friction points that you start overlooking the longer you stare at them.
“Paper buddies”
Lucas:
I think that’s something that Viola also tries to push a bit now that we have these “paper buddies”: co-authors whose task is to help that paper and to get the whole project become clear. You get acknowledged by being co-author but essentially you’re like an editor for that paper and make sure that it is clear and so on. I did that with Matthias and that was extremely helpful for him to have someone that is not the supervisor but who knows about the project and can help with the communication parts.
Paul:
If you want to really contribute to making the text better, you have to understand the content. So it’s the ask of the primary author to explain to you what is happening and ideally do it through his text. And if the text doesn’t communicate it then it’s your task to find out and then make the text better. That way everyone ends up understanding the whole thing much better. I think it’s great! I do this with everything and I usually pick Johannes. And because we are well co-aligned with projects we know about each other’s project and so you have like your paper buddy for everything. It’s really useful.
Collaborative writing and its inherent problems
Martina:
Nice! Sounds good! So you are productive group now, yeah? Writing a lot of papers…
In collaborative writing you have multiple authors consequently multiple opinions and styles.
Lucas:
If one has these close collaborations then good collaborative writing is still a challenge. It’s one thing to have one author that writes the paper and then you just give feedback. This is a lengthy process and can be frustrating because you only give suggestions — in the end it’s the first author that decides what to take and not. And sometimes you’re frustrated because you think that solution would be really better and then this will not be incorporated. On the other hand if you become too invasive and you edit the text, then people also get frustrated. I haven’t found out the most efficient way to do this, to be honest.
Paul:
I would argue that there is no efficient way. This is inherently the problem: that you have multiple authors consequently multiple opinions and styles. If you want to have a consistent paper that could only be one person that writes but that’s exhausting for the person. The other option would be to compromise on a consistent style and then you can easily or easier compromise on paragraphs. So I have this problem that when I see a text and I immediately know what I would do differently, it usually boils down to sentence structure and how to split paragraphs. And that’s something that’s not so super invasive because the content stays mostly the same and the message stays the same. But it’s nice to have these two versions of the text side by side and then the main author can pick what he likes.
Lucas:
I think that’s also in the end what we settled with. But there were situations where I just changed formulations and this was really a no-go. Or at least without proper explaining what happened. So this is really a part where it becomes too invasive if it’s about these small things.
Let go. It’s all much nicer and much easier if you don’t care too much.
Paul:
Sure, especially it’s these little nitty details where you fiddle a long time and then you have this one formulation nailed and you think that’s the one and the rest of the paragraph really doesn’t matter. It’s all just this one tiny phrase. And author number two comes along and throws that one phrase out because he doesn’t like it…
Martina:
Maybe it just doesn’t fit. Often you fiddle with one sentence and you forget the rest of the paragraph then when you see it, it just doesn’t fit. And you are like: “whatever, I love the one sentence, I keep it!”
Paul:
What really helped me in that sense is to let go. That’s something I keep discovering every time I’m done with a text. It’s all much nicer and much easier if you don’t care too much. Yeah, let go of it…
Giving a talk as early feedback and “lean PhD”
Martina:
Since I have been the last time at your group, I developed a more nuanced approach to the feedback: you start with the feedback before you start writing. First, you present only the figures with captions and tell a story and get feedback on that, whether this makes sense. Then the next step is to give a talk. In the talk you also have an introduction, you should have some discussion. You will get questions. You get a lot of feedback and only then you start writing the first draft. You are already at the level that you can’t have some major stuff wrong because you already got so much feedback before you wrote the first sentence.
Paul:
I feel that this would benefit the scientific community, this basically thinking the other way around. Usually what we do is we work, we finish it, we write it down, and then we give a talk about it at a conference.And that’s bad because you have all this tedious pipeline and you don’t get feedback on time. But if you make a talk about something that you do not feel secure about it’s hard to do but it will help you and you’re the main one that should benefit from such a type of talk. But i don’t know what the perfect audience for such a talk would be…
Martina:
… your group, I guess. And maybe the neighboring group. You know, something like an institute colloquium. I think there are issues about presenting this on a conference: it’s unpublished… Maybe you handpick a couple of people who know about this stuff if it’s something interdisciplinary and people from your group are not the best experts…
Paul:
Sometimes that just happens accidentally, like just last week: I asked Lucas and Jonas for some help with something that I am facing. And just to give them context to to go to the point where they can answer the question I had, I had to know summarize what we do and that way you get also content questions that you cannot ask yourself. That’s nice — it’s exactly the type of feedback you need to develop the project in your mind.
Let go of all the fancy “toppings”. It’s about clarity and early iterations.
Lucas:
That’s kind of integral question for the whole scientific process. There’s this one idea put forward by Julian Kirchherr who wrote the book “The lean PhD”. You try to use a technique from startups called rapid prototyping. And the minimal viable product, you want to push that out as soon as possible to get feedback on your on your project or your ideas and then you can incorporate that and iterate. You want to get to that point that you can iterate as fast as possible. I think the key ingredient for these iterations to work is clarity. So it’s not so what I always did when I wrote my texts initially. I thought they would have to be beautiful or fancy sounding, so I used words that I found gave the whole thing more weight but in the end they just made it unclear. What it’s about: people have to understand what you’re doing and you have to be clear about what do I actually want to do with this project. And then there’s little points where it can go wrong because people give you the feedback that you need and then you improve. And this way it gets really better. You have to let go of all the fancy “toppings” and really it’s about clarity and early iterations.
Paul:
That’s a very difficult and very high-level understanding that that you have gained already. It takes a year to get there. That’s great.
Project design with feedback
With feedback you explore in all directions.
Lucas:
We try to practice that even now from the project design. We have this one group meeting where we focus on a small topic and we have a lot of master students that started their thesis. They were given really broad topics and we found that it’s really hard to define a precise project. That’s essentially what you should start with before you even start a simulation: you want to be clear with what is my goal. Just doing this once with peers is super helpful, so we forced each other to be really specific about what’s the project goal and for a lot of them it became really: “okay, this is what I want to focus on”. And it’s much easier to work like this than just exploration.
Paul:
Because you explore in all directions. If you don’t have this early feedback you go and run in a random direction and you realize that was pointless and then you turn around and run again.
Why peer feedback is hard
Martina:
So working with peers is really helpful, as you are saying, as you are demonstrating with your work. Now what do you think why is it so hard to convince people to give it a try?
When you open up for feedback you run into the danger of being exposed of not being on top of your game.
Lucas:
It’s really hard. [laughs] As Paul said, just to be able to get our feedback, he had to put a lot of work into preparing it. And it’s kind of nasty work because you think — that’s what our brain suggests us — that you’re on top of it: “okay, I made these plots, I know why, I understand the things so why do I have to now put these correct axes labels. I know what’s in there… and why do I have to make the thing understandable…” Also this question why am I actually doing what I’m doing — it’s a nasty question if you really follow it to the cause. But it’s the important questions… I think it’s just the convenience, it’s just very hard to do, that’s why we shy away from it.
Paul:
And you kind of expose yourself, right? It’s on the other end of the spectrum: on the one hand you feel like you’re on top of your game as you suggested but when you open up for feedback you run into the danger of being exposed of not being on top of your game. So that’s totally not intuitive for us and we have to just let go, right? That’s again, letting go is hard…
Lucas:
And the longer you wait the harder it becomes. If you worked two weeks you just thought about: “okay, what’s my project about” and then you know you just put: “okay, I had these ideas but I’m really uncertain about them”. Then it’s a different story then after half a year you come and you know actually I have no idea what I’m doing. It just becomes more painful the more you wait.
Paul’s experience with the fast-paced writing of their Covid paper published in Science
Martina:
I wanted to talk about this Covid paper published in Science. I saw it, it came to me from my social network somehow. So this was a very fast one. I think, Paul; you were on that, right? So how was that? The data is from March, the paper was submitted in April, right? And in May it was out!
Paul:
So, it was exhausting because… this is difficult to answer… On one hand it was exhausting, on the other hand it was great because you had a lot of people working on the same thing and it was the top priority for everyone. That is usually not the case on other projects. But this only happened because we knew that it is timely and there is a hard deadline. So in terms of the science it was quite iterative and there was a lot of fine-tuning while text was already produced. So that’s something one should maybe implement in everyday work too: that you start writing just for yourself in a more relaxed context to have something to work with and this will enable you to to follow your own thoughts later much nicer. So that’s the whole lab book idea. But here it was more fast-paced and distributed among people.
Now what turned out to be difficult is how to collaborate with people on text and to improve text in these fast-paced iterations. You have these online text editors like CodiMD where everyone can work on the same document in real time or share it — and then, all of a sudden, you start standing on each each other’s feet and it’s difficult to work around this.
One take-away insight was to assign an editor in charge who is responsible for the whole document: he delegates sections but in the end he makes the decisions. In this case this was Michael Wilczek who was not the senior author in that sense because Viola managed the whole project but he was the text managing director, And that was very useful, that helped.
Martina:
That helped, I can imagine, especially when the tempo is fast. So, all co-authors were writing something? Or most of them?
What turned out to be difficult is how to collaborate with people on text and improve text in these fast-paced iterations.
Paul:
Yeah. Everyone was writing little sections and Johannes and I wrote a discussion early on that just went to the trash at some point. It’s just what happens, right? You write a discussion and someone else writes discussion. You have them side by side and you pick the points that are important. And sometimes it just turns out to be a different project altogether… So it turned out that the project just went elsewhere. We started off with simple prediction and we realized we want to make this more complicated, or to capture more effects. And corrections were added and then we realized that the discussion just does not relate to the original article on the work anymore. Then you have to rewrite it. Then it helps to let go.
Martina:
Yeah.. but I bet that because you wrote the “wrong” discussion, that help you to get faster to the right one.
Paul:
Definitely. Also for Johannes and me it was nice to see what’s important to us and then to see how that is important to people that know the field. These are more from this bayesian context for whom this was completely crystal clear so we were really discussing on obvious points and: okay, no need for that in many cases.
Martina:
I see… Yeah, that can happen.
Paul:
But to summarize this was amazing, with great insight. I’m very happy that I was able to take part in this. We all learned a lot and it was an impactful experience.
Martina:
Hey, nice, I was very happy to see that paper and when I saw it I immediately looked in and, yeah, I think it’s it’s well written.
Advice to other researchers
Let it go!
Martina:
… okay I think we talked about the most important stuff, maybe let’s slowly wrap it up. So, do you have some final advice? You already gave a lot of advice but when you could just think about one thing to tell people who are starting working on their writing skills, maybe trying out the peer feedback. What would be just one piece of advice that made the biggest difference for you?
Lucas:
I think for me it’s: enjoy the process and… yeah, let it go.
Paul:
Yeah, plus one.
Martina:
Thank you so much for your time and for sharing your experiences and your insights. It will be useful to other people as well. Thank you so much!