A referee report is not meant to be a review or a reaction to an academic article, but an assessment. Your role as referee is to provide a judgement as to whether the article has been executed well, and whether it is convincing. You are providing a recommendation to the editor, and not a decision. As an input you may find this Research Assessment helpful.
If your recommendation is REJECT you should provide:
- 2 clear paragraphs for the editor
- 5 suggestions for the author
If your recommendation is R&R you should provide:
- A clear explanation of what you consider to be the minimum requirements to get the paper over the line
- Thoughts on how to improve the article beyond this
- Note that you have the benefit of anonymity so you can be harsher than the editor and make their job easier.
- Brief synopsis – write a single paragraph (2 at most) summarising the article in your own words. Do not merely repeat the abstract. Either demonstrate that you understood the article or explain what your didn’t understand about it.
- Assessment – provide a summary of your opinion of the paper. Assess how the paper fits into the broader literature. Be clear on any points that you don’t feel qualified to judge.
- Areas/issues – provide detail on perceived weaknesses. Smaller issues (e.g. typos) shouldn’t form the bulk of the report.
Much of this advice is based on Ed Glaeser’s talk at the 2019 EEA Meetings.