essays have references

How to Write an Academic Essay with References and Citations

#scribendiinc

Written by  Scribendi

If you're wondering how to write an academic essay with references, look no further. In this article, we'll discuss how to use in-text citations and references, including how to cite a website, how to cite a book, and how to cite a Tweet, according to various style guides.

You might need to cite sources when writing a paper that references other sources. For example, when writing an essay, you may use information from other works, such as books, articles, or websites. You must then inform readers where this information came from. Failure to do so, even accidentally, is plagiarism—passing off another person's work as your own.

You can avoid plagiarism and show readers where to find information by using citations and references. 

Citations tell readers where a piece of information came from. They take the form of footnotes, endnotes, or parenthetical elements, depending on your style guide. In-text citations are usually placed at the end of a sentence containing the relevant information. 

A reference list , bibliography, or works cited list at the end of a text provides additional details about these cited sources. This list includes enough publication information allowing readers to look up these sources themselves.

Referencing is important for more than simply avoiding plagiarism. Referring to a trustworthy source shows that the information is reliable. Referring to reliable information can also support your major points and back up your argument. 

Learning how to write an academic essay with references and how to use in-text citations will allow you to cite authors who have made similar arguments. This helps show that your argument is objective and not entirely based on personal biases.

How Do You Determine Which Style Guide to Use?

Often, a professor will assign a style guide. The purpose of a style guide is to provide writers with formatting instructions. If your professor has not assigned a style guide, they should still be able to recommend one. 

If you are entirely free to choose, pick one that aligns with your field (for example, APA is frequently used for scientific writing). 

Some of the most common style guides are as follows:

AP style for journalism

Chicago style for publishing

APA style for scholarly writing (commonly used in scientific fields)

MLA style for scholarly citations (commonly used in English literature fields)

Some journals have their own style guides, so if you plan to publish, check which guide your target journal uses. You can do this by locating your target journal's website and searching for author guidelines.

How Do You Pick Your Sources?

When learning how to write an academic essay with references, you must identify reliable sources that support your argument. 

As you read, think critically and evaluate sources for:

Objectivity

Keep detailed notes on the sources so that you can easily find them again, if needed.

Tip: Record these notes in the format of your style guide—your reference list will then be ready to go.

How to Use In-Text Citations in MLA

An in-text citation in MLA includes the author's last name and the relevant page number: 

(Author 123)

How to Cite a Website in MLA

Here's how to cite a website in MLA:

Author's last name, First name. "Title of page."

Website. Website Publisher, date. Web. Date

retrieved. <URL>

With information from a real website, this looks like:

Morris, Nancy. "How to Cite a Tweet in APA,

Chicago, and MLA." Scribendi. Scribendi

Inc., n.d. Web. 22 Dec. 2021.

<https://www.scribendi.com/academy/articles/how_to_cite_a_website.en.html>

How Do You Cite a Tweet in MLA ?

MLA uses the full text of a short Tweet (under 140 characters) as its title. Longer Tweets can be shortened using ellipses. 

MLA Tweet references should be formatted as follows:

@twitterhandle (Author Name). "Text of Tweet." Twitter, Date Month, Year, time of

publication, URL.

With information from an actual Tweet, this looks like:

@neiltyson (Neil deGrasse Tyson). "You can't use reason to convince anyone out of an

argument that they didn't use reason to get into." Twitter, 29 Sept. 2020, 10:15 p.m.,

https://twitter.com/neiltyson/status/1311127369785192449 .

How to Cite a Book in MLA

Here's how to cite a book in MLA:

Author's last name, First name. Book Title. Publisher, Year.

With publication information from a real book, this looks like:

Montgomery, L.M. Rainbow Valley. Frederick A. Stokes Company, 1919.

How to Cite a Chapter in a Book in MLA

Author's last name, First name. "Title of Chapter." Book Title , edited by Editor Name,

Publisher, Year, pp. page range.

With publication information from an actual book, this looks like:

Ezell, Margaret J.M. "The Social Author: Manuscript Culture, Writers, and Readers." The

Broadview Reader in Book History , edited by Michelle Levy and Tom Mole, Broadview

Press, 2015,pp. 375–394.

How to  Cite a Paraphrase in MLA

You can cite a paraphrase in MLA exactly the same way as you would cite a direct quotation. 

Make sure to include the author's name (either in the text or in the parenthetical citation) and the relevant page number.

How to Use In-Text Citations in APA

In APA, in-text citations include the author's last name and the year of publication; a page number is included only if a direct quotation is used: 

(Author, 2021, p. 123)

How to Cite a Website in APA

Here's how to cite a website in APA:

Author, A. A., & Author, B. B. (Year, Month. date of publication). Title of page. https://URL

Morris, N. (n.d.). How to cite a Tweet in APA, Chicago, and MLA. 

https://www.scribendi.com/academy/articles/how_to_cite_a_website.en.html       

Tip: Learn more about how to write an academic essay with  references to websites .

How Do You  Cite a Tweet in APA ?

APA refers to Tweets using their first 20 words. 

Tweet references should be formatted as follows:

Author, A. A. [@twitterhandle). (Year, Month. date of publication). First 20 words of the

Tweet. [Tweet] Twitter. URL

When we input information from a real Tweet, this looks like:

deGrasse Tyson, N. [@neiltyson]. (2020, Sept. 29). You can't use reason to convince anyone

out of an argument that they didn't use reason to get into. [Tweet] Twitter.

https://twitter.com/neiltyson/status/1311127369785192449

How to Cite a Book in APA

Here's how to cite a book in APA:   

Author, A. A. (Year). Book title. Publisher.

For a real book, this looks like:

Montgomery, L. M. (1919). Rainbow valley.

Frederick A. Stokes Company.

How to Cite a Chapter in a Book in APA

Author, A. A. (Year). Chapter title. In Editor Name (Ed.), Book Title (pp. page range).

With information from a real book, this looks like:

Ezell, M. J. M. (2014). The social author: Manuscript culture, writers, and readers. In

Michelle Levy and Tom Mole (Eds.), The Broadview Reader in Book History (pp. 375–

394). Broadview Press.

Knowing how to cite a book and how to cite a chapter in a book correctly will take you a long way in creating an effective reference list.

How to Cite a Paraphrase in APA

You can cite a paraphrase in APA the same way as you would cite a direct quotation, including the author's name and year of publication. 

In APA, you may also choose to pinpoint the page from which the information is taken.

Referencing is an essential part of academic integrity. Learning how to write an academic essay with references and how to use in-text citations shows readers that you did your research and helps them locate your sources.

Learning how to cite a website, how to cite a book, and how to cite a paraphrase can also help you avoid plagiarism —an academic offense with serious consequences for your education or professional reputation.

Scribendi can help format your citations or review your whole paper with our Academic Editing services .

Take Your Essay from Good to Great

Hire an expert academic editor , or get a free sample, about the author.

Scribendi's in-house editors work with writers from all over the globe to perfect their writing. They know that no piece of writing is complete without a professional edit, and they love to see a good piece of writing transformed into a great one. Scribendi's in-house editors are unrivaled in both experience and education, having collectively edited millions of words and obtained numerous degrees. They love consuming caffeinated beverages, reading books of various genres, and relaxing in quiet, dimly lit spaces.

Have You Read?

"The Complete Beginner's Guide to Academic Writing"

Related Posts

APA Style and APA Formatting

How to Research a Term Paper

MLA Formatting and MLA Style: An Introduction

Upload your file(s) so we can calculate your word count, or enter your word count manually.

We will also recommend a service based on the file(s) you upload.

English is not my first language. I need English editing and proofreading so that I sound like a native speaker.

I need to have my journal article, dissertation, or term paper edited and proofread, or I need help with an admissions essay or proposal.

I have a novel, manuscript, play, or ebook. I need editing, copy editing, proofreading, a critique of my work, or a query package.

I need editing and proofreading for my white papers, reports, manuals, press releases, marketing materials, and other business documents.

I need to have my essay, project, assignment, or term paper edited and proofread.

I want to sound professional and to get hired. I have a resume, letter, email, or personal document that I need to have edited and proofread.

 Prices include your personal % discount.

 Prices include % sales tax ( ).

• PRO Courses Guides New Tech Help Pro Expert Videos About wikiHow Pro Upgrade Sign In
• EDIT Edit this Article
• EXPLORE Tech Help Pro About Us Random Article Quizzes Request a New Article Community Dashboard This Or That Game Popular Categories Arts and Entertainment Artwork Books Movies Computers and Electronics Computers Phone Skills Technology Hacks Health Men's Health Mental Health Women's Health Relationships Dating Love Relationship Issues Hobbies and Crafts Crafts Drawing Games Education & Communication Communication Skills Personal Development Studying Personal Care and Style Fashion Hair Care Personal Hygiene Youth Personal Care School Stuff Dating All Categories Arts and Entertainment Finance and Business Home and Garden Relationship Quizzes Cars & Other Vehicles Food and Entertaining Personal Care and Style Sports and Fitness Computers and Electronics Health Pets and Animals Travel Education & Communication Hobbies and Crafts Philosophy and Religion Work World Family Life Holidays and Traditions Relationships Youth
• Browse Articles
• Learn Something New
• Quizzes Hot
• This Or That Game New
• Train Your Brain
• Explore More
• Support wikiHow
• About wikiHow
• Log in / Sign up
• Education and Communications
• College University and Postgraduate
• Academic Writing

How to Reference Essays

Last Updated: January 8, 2024 Fact Checked

This article was co-authored by Alexander Peterman, MA . Alexander Peterman is a Private Tutor in Florida. He received his MA in Education from the University of Florida in 2017. There are 14 references cited in this article, which can be found at the bottom of the page. This article has been fact-checked, ensuring the accuracy of any cited facts and confirming the authority of its sources. This article has been viewed 366,192 times.

When you begin writing a research essay, you must take into account the format of your writing and reference pages. There are several reference styles that may be assigned to you, including MLA (Modern Language Association), APA (American Psychological Association), and Chicago. Each one has its own set of rules. There's no need to familiarize yourself with all 3 unless you have to, but you do need to learn at least one if you’re in any field involving academic writing. Here are summaries of each style to help you start your essay on the right track.

Referencing Essays Templates

• You will need a citation directly after every sentence (or group of sentences if you're citing the same source in multiple consecutive sentences) containing information you didn't think of yourself. These include: paraphrases, facts, statistics, quotes, and examples.
• An in-text citation using MLA will simply have the author last name (or title if no author) followed by the page number. No comma between author and page number. For example: (Richards 456) Richards is the author last name, and 456 is the page number.
• If you have an author name (or title, if no author) but no page number, simply use author last name (or title).

• The easiest way to keep track of MLA citations while doing research is to copy and paste copyright information into a word processing document as you go, or to write it down in a notebook.
• Things to include for any source are author(s), date published, publisher, page number, volume and issue number, website, date accessed, anything that appears on the copyright page or indicates how to find it again. [2] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source

• As an example, the format for a standard book citation using MLA style is as follows: Last name of author, First name. Title of Book. City published: Publisher Name, Year published. Source Medium.
• An MLA website citation looks like the following. If there's no author listed, begin citation with the name of the page: Last name, first name. "Page Title." Website Title. Publisher. Date published. Source Medium. Date accessed.
• An MLA scholarly article citation looks like the following: Last name, First name. "Title of Article." Title of Journal . Volume.Issue (Year): page numbers. Source Medium.
• Write the title of the main work (book, magazine, journal, website, etc.) in italics, or underline if you’re writing references by hand.
• Chapter or article titles should be in quotation marks.

• If there is no author listed, as is common on websites, simply skip the author’s name and begin the entry with the title of the work.
• Alphabetize by the first letter that appears in the entry, whether it has an author name or not.

• The formatting should be in Times New Roman font, size 12, with “Works Cited” centered at the top of a new page.
• Each entry should have hanging indent, meaning all lines below the first line are indented by half an inch.
• Make sure there is a period after each section of the citations. A period should always end the citation.

• Place a parenthetical citation at the end of every sentence (or group of sentences if you're using the same source for multiple consecutive sentences) containing information you didn't know before doing research.
• An in-text citation using APA will simply have the author last name (or title if no author) followed by the year it was published. No comma between name and year. For example: (Richards 2005) Richards is the author last name, and 2005 is the year.
• If you have an author name (or title if no author) but no page number, simply use author last name (or title). This is common when citing websites.
• APA document formatting is very important. APA papers are divided up into 4 sections: the title page, the abstract, the main body, and the references page. The citations of a research paper using APA appear in the References section, the last portion of an APA document. [7] X Research source

• To form APA reference page citations, you will need such information as author name(s), date published, website URL, date you accessed the website, title of work, and so on. [8] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source

• For example, the format for an APA reference of a scholarly journal article is as follows: Author last name, First initial. (Year published). Article or chapter title. Journal or book title, Issue number , page number range. [10] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source
• The format for an APA book reference looks like: Last name, First name. (Year.) Title of Book: Capital letter also for subtitle . Location: Publisher.
• The format for an APA website reference looks like: Author, A.A. First name, & Author, B.B. (Date published.) Title of article. In Title of webpage or larger document or book (chapter or section number). Retrieved from URL address

• Capitalize the author's last name and first initial, followed by a period.
• Only capitalize the first word of a journal article title, unless the title contains a proper noun (called sentence case). Titles of books should preserve the published capitalization.
• Capitalize the city of publication, and use correct state abbreviations for states. Also capitalize the name of the publisher and end the reference with a period.
• The title of larger works, whether a book, journal, website, or magazine, is in italics (or underlined if handwriting), as is the issue number that appears right after the title. Titles for shorter works like articles and chapters should not have any indicative punctuation in an APA entry. [12] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source
• A period should end all citations.

Using Chicago Manual of Style

• For Notes and Bibliography, you will use a superscript at the instance of each quote in the text with a corresponding footnote at the end of the page. All footnotes are compiled into endnotes at the end of the work, on the bibliography page. [14] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source
• For Author Date, you will use parenthetical in-text citations that include author last name and year published, using no punctuation between name and year. The full version of each parenthetical citation is listed alphabetically on the references page. For example: (Simon 2011) Simon is the author last name, and 2011 is the year.
• You will need a citation directly after every sentence (or group of sentences if you're using the same source for multiple consecutive sentences) containing information you didn't think of yourself. These include: paraphrases, facts, statistics, quotes, and examples.

• If using a book, write down all pertinent information found on the copyright page, including the name of the publisher and the city and year of publication.
• For other sources, look for this information near the title of the piece you’re looking at. Publication date is often at the bottom of webpages.

• Title your references page “Bibliography” centered at the top of the page. Leave 2 blank lines between this title and the first entry, and one blank line between entries.
• Notes and Bibliography style uses footnotes for page endings and endnotes for chapter endings. The bibliography page will be an alphabetized list of all sources in hanging indent.
• An example format for a book is as follows: Last name, First name. Book Title . City: Publisher, Year.
• An example format for a chapter in a print scholarly journal is as follows: Author last name, first name. "Title of Chapter or Article." Book or journal Title Issue Number (Year): Page number range. (For an online scholarly journal article, tack on the following at the end: Date accessed. URL address.)
• When there is no known author, the entry should begin with the title of the document, whether it's a webpage, chapter, article, and so on.
• When there are multiple authors, the first listed author appears last name, first name, so that the citation is alphabetized by this author's last name. Subsequent authors are listed by first name, like this: Alcott, Louisa May, Charles Dickens, and Elizabeth Gaskell.
• Always end a citation with a period.

• When using Author Date style, title your references page “References” centered at the top of the page. Leave 2 blank lines between this title and the first entry, and 1 blank line between entries.
• Author Date style bibliographies should be organized alphabetically by last name (or by title if no author) in hanging indent.
• An example format for a book is as follows: Last name, first name. Year. Book Title . City Published: Publisher.
• An example format for a chapter in a print scholarly journal is as follows: Author last name, first name. Year. "Title of Chapter or Article." Book or journal title issue number: page numbers. (for an online scholarly journal article tack this onto the end: Date accessed. URL address.)
• An example format for a website is as follows: Name of Website. Year. "Page Title." Date last modified. Date accessed. URL address.

Expert Q&A

• You don't have to write each bibliography or reference entry on your own. You can download citation management software like Endnote [17] X Research source (purchase required on this one), Zotero [18] X Research source (it's free), or use websites like http://www.bibme.org/ and http://www.easybib.com/ . Select the name of your style manual before you begin creating citations. Copy and paste the citation into your bibliography or references list. Thanks Helpful 0 Not Helpful 0
• If you are assigned to write a paper or other written document in one of these styles, you need to purchase the style manual. It will contain nearly every instance not only of source citation, but paper formatting as well as grammar and punctuation that is unique to that style. Thanks Helpful 0 Not Helpful 0

• This article only lists how to cite research for each style manual. Each style has its own instructions for setting up the format of the essay, including heading, spacing, margins, font, and so on. Thanks Helpful 1 Not Helpful 0

You Might Also Like

• ↑ https://owl.purdue.edu/owl/research_and_citation/mla_style/mla_formatting_and_style_guide/mla_in_text_citations_the_basics.html
• ↑ https://owl.purdue.edu/owl/research_and_citation/mla_style/mla_formatting_and_style_guide/mla_formatting_and_style_guide.html
• ↑ https://owl.purdue.edu/owl/research_and_citation/mla_style/mla_formatting_and_style_guide/mla_works_cited_page_books.html
• ↑ https://owl.purdue.edu/owl/research_and_citation/apa_style/apa_formatting_and_style_guide/in_text_citations_author_authors.html
• ↑ https://owl.english.purdue.edu/owl/resource/560/01/
• ↑ https://owl.purdue.edu/owl/research_and_citation/apa_style/apa_formatting_and_style_guide/reference_list_electronic_sources.html
• ↑ https://libguides.jcu.edu.au/apa/reference-list
• ↑ https://owl.purdue.edu/owl/research_and_citation/apa_style/apa_formatting_and_style_guide/reference_list_author_authors.html
• ↑ https://owl.purdue.edu/owl/research_and_citation/apa_style/apa_formatting_and_style_guide/reference_list_basic_rules.html
• ↑ https://www.chicagomanualofstyle.org/tools_citationguide.html
• ↑ https://owl.purdue.edu/owl/research_and_citation/chicago_manual_17th_edition/cmos_formatting_and_style_guide/chicago_manual_of_style_17th_edition.html
• ↑ http://guides.nyu.edu/c.php?g=276562&p=1844734
• ↑ http://endnote.com
• ↑ https://www.zotero.org

About This Article

To reference an essay using MLA style, add a citation after any information you found through a source, like facts or quotes. When citing the reference, include the author’s name and the page number you pulled the information from in parenthesis, like “(Richards 456).” Once you’ve finished your essay, add a Words Cited page with all of the information you used to research your essay, like books or articles. To create a Works Cited page, list the sources in alphabetical order using the author’s last name, and include additional information, like year published and the medium. For more tips from our Writing reviewer, like how to reference an essay using APA style, read on! Did this summary help you? Yes No

• Send fan mail to authors

Reader Success Stories

Alicia Hammond

Sep 20, 2021

Did this article help you?

Oct 30, 2019

Brenda Nats

Jun 14, 2021

Featured Articles

Trending Articles

Watch Articles

• Terms of Use
• Privacy Policy
• Do Not Sell or Share My Info
• Not Selling Info

wikiHow Tech Help Pro:

Level up your tech skills and stay ahead of the curve

How to Reference in an Essay (9 Strategies of Top Students)

Are you feeling overwhelmed by referencing?

When you’re first asked to do referencing in an essay it can be hard to get your head around it. If it’s been a while since you were first taught how to reference, it can be intimidating to ask again how to do it!

I have so many students who consistently lose marks just because they didn’t get referencing right! They’re either embarrassed to ask for extra help or too lazy to learn how to solve the issues.

So, here’s a post that will help you solve the issues on your own.

Already think you’re good at referencing? No worries. This post goes through some surprising and advanced strategies for anyone to improve no matter what level you are at!

In this post I’m going to show you exactly how to reference in an essay. I’ll explain why we do it and I’ll show you 9 actionable tips on getting referencing right that I’m sure you will not have heard anywhere else!

The post is split into three parts:

• What is a Reference and What is a Citation?
• Why Reference? (4 Things you Should Know)
• How to Reference (9 Strategies of Top Students)

If you think you’ve already got a good understanding of the basics, you can jump to our 9 Advanced Strategies section.

Part 1: What is a Reference and What is a Citation?

What is a citation.

An in-text mention of your source. A citation is a short mention of the source you got the information from, usually in the middle or end of a sentence in the body of your paragraph. It is usually abbreviated so as not to distract the reader too much from your own writing. Here’s two examples of citations. The first is in APA format. The second is in MLA format:

• APA: Archaeological records trace the original human being to equatorial Africa about 250,000–350,000 years ago (Schlebusch & Jakobsson, 2018) .
• MLA: Archaeological records trace the original human being to equatorial Africa about 250,000–350,000 years ago (Schlebusch and Jakobsson 1) .

In APA format, you’ve got the authors and year of publication listed. In MLA format, you’ve got the authors and page number listed. If you keep reading, I’ll give some more tips on formatting further down in this article.

And a Reference is:

What is a Reference?

A reference is the full details of a source that you list at the end of the article. For every citation (see above) there needs to be a corresponding reference at the end of the essay showing more details about that source. The idea is that the reader can see the source in-text (i.e. they can look at the citation) and if they want more information they can jump to the end of the page and find out exactly how to go about finding the source.

Here’s how you would go about referencing the Schlebusch and Jakobsson source in a list at the end of the essay. Again, I will show you how to do it in APA and MLA formats:

• APA: Schlebusch, C. & Jakobsson, M. (2018). Tales of Human Migration, Admixture, and Selection in Africa. Annual Review of Genomics and Human Genetics , 11 (33), 1–24.
• MLA: Schlebusch, Carina and Mattias Jakobsson. “Tales of Human Migration, Admixture, and Selection in Africa.” Annual Review of Genomics and Human Genetics , vol. 11, no. 33, 2018, pp. 1–24.

In strategy 1 below I’ll show you the easiest and fool proof way to write these references perfectly every time.

One last quick note: sometimes we say ‘reference’ when we mean ‘citation’. That’s pretty normal. Just roll with the punches. It’s usually pretty easy to pick up on what our teacher means regardless of whether they use the word ‘reference’ or ‘citation’.

Part 2: Why Reference in an Essay? (4 Things you Should Know)

Referencing in an essay is important. By the time you start doing 200-level courses, you probably won’t pass the course unless you reference appropriately. So, the biggest answer to ‘why reference?’ is simple: Because you Have To!

Okay let’s be serious though … here’s the four top ‘real’ reasons to reference:

1. Referencing shows you Got an Expert’s Opinion

You can’t just write an essay on what you think you know. This is a huge mistake of beginning students. Instead this is what you need to do:

Top Tip: Essays at university are supposed to show off that you’ve learned new information by reading the opinions of experts.

Every time you place a citation in your paragraph, you’re showing that the information you’re presenting in that paragraph was provided to you by an expert. In other words, it means you consulted an expert’s opinion to build your knowledge.

If you have citations throughout the essay with links to a variety of different expert opinions, you’ll show your marker that you did actually genuinely look at what the experts said with an open mind and considered their ideas.

This will help you to grow your grades.

2. Referencing shows you read your Assigned Readings

Your teacher will most likely give you scholarly journal articles or book chapters to read for homework between classes. You might have even talked about those assigned readings in your seminars and tutorials.

Great! The assigned readings are very important to you.

You should definitely cite the assigned readings relevant to your essay topic in your evaluative essay (unless your teacher tells you not to). Why? I’ll explain below.

• Firstly, the assigned readings were selected by your teacher because your teacher (you know, the person who’s going to mark your essay) believes they’re the best quality articles on the topic. Translation: your teacher gave you the best source you’re going to find. Make sure you use it!
• Secondly, by citing the assigned readings you are showing your teacher that you have been paying attention throughout the course. You are showing your teacher that you have done your homework, read those assigned readings and paid attention to them. When my students submit an essay that has references to websites, blogs, wikis and magazines I get very frustrated. Why would you cite low quality non-expert sources like websites when I gave you the expert’s article!? Really, it frustrates me so, so much.

So, cite the assigned readings to show your teacher you read the scholarly articles your teacher gave to you. It’ll help you grow your marks.

3. Referencing deepens your Knowledge

Okay, so you understand that you need to use referencing to show you got experts’ opinions on the topic.

But there’s more to it than that. There’s actually a real benefit for your learning.

If you force yourself to cite two expert sources per paragraph, you’re actually forcing yourself to get two separate pieces of expert knowledge. This will deepen your knowledge!

So, don’t treat referencing like a vanity exercise to help you gain more marks. Actually view it as an opportunity to develop deeper understandings of the topic!

When you read expert sources, aim to pick up on some new gems of knowledge that you can discuss in your essays. Some things you should look out for when finding sources to reference:

• Examples that link ideas to real life. Do the experts provide real-life examples that you can mention in your essay?
• Facts and figures. Usually experts have conducted research on a topic and provide you with facts and figures from their research. Use those facts and figures to deepen your essay!
• Short Quotes. Did your source say something in a really interesting, concise or surprising way? Great! You can quote that source in your essay .
• New Perspectives. Your source might give you another perspective, angle or piece of information that you can add to your paragraph so that it’s a deep, detailed and interesting paragraph.

So, the reason we ask you to reference is at the end of the day because it’s good for you: it helps you learn!

4. Referencing backs up your Claims

You might think you already know a ton of information about the topic and be ready to share your mountains of knowledge with your teacher. Great!

So, should you still reference?

Yes. Definitely.

You need to show that you’re not the only person with your opinion. You need to ‘stand on the shoulders of giants.’ Show what other sources have said about your points to prove that experts agree with you.

You should be saying: this is my opinion and it’s based on facts, expert opinions and deep, close scrutiny of all the arguments that exist out there .

If you make a claim that no one else has made, your teacher is going to be like “Have you even been reading the evidence on this topic?” The answer, if there are no citations is likely: No. You haven’t.

Even if you totally disagree with the experts, you still need to say what their opinions are! You’ll need to say: “This is the experts’ opinions. And this is why I disagree.”

So, yes, you need to reference to back up every claim. Try to reference twice in every paragraph to achieve this.

Part 3: Strategies for How to Reference in an Essay (9 Strategies of Top Students)

Let’s get going with our top strategies for how to reference in an essay! These are strategies that you probably haven’t heard elsewhere. They work for everyone – from beginner to advanced! Let’s get started:

1. Print out your Reference Style Cheat Sheet

Referencing is hard and very specific. You need to know where to place your italics, where the commas go and whether to use an initial for full name for an author.

There are so many details to get right.

And here’s the bad news: The automated referencing apps and websites nearly always get it wrong! They tell you they can generate the citation for you. The fact of the matter is: they can’t!

Here’s the best way to get referencing right: Download a referencing cheat sheet and have it by your side while writing your essay.

Your assignment outline should tell you what type of referencing you should use. Different styles include: APA Style, MLA Style, Chicago Style, Harvard Style, Vancouver Style … and many more!

You need to find out which style you need to use and download your cheat sheet. You can jump onto google to find a cheat sheet by typing in the google bar:

Download a pdf version of the referencing style cheat sheet, print it out, and place it on your pinboard or by your side when writing your essay.

2. Only cite Experts

There are good and bad sources to cite in an essay.

You should only cite sources written, critiqued and edited by experts. This shows that you have got the skill of finding information that is authoritative. You haven’t just used information that any old person popped up on their blog. You haven’t just gotten information from your local newspaper. Instead, you got information from the person who is an absolute expert on the topic.

Here’s an infographic listing sources that you should and shouldn’t cite. Feel free to share this infographic on social media, with your teachers and your friends:

3. Always use Google Scholar

Always. Use. Google. Scholar.

Ten years ago students only had their online university search database to find articles. Those university databases suck. They rarely find the best quality sources and there’s always a big mix of completely irrelevant sources mixed in there.

Google Scholar is better at finding the sources you want. That’s because it looks through the whole article abstract and analyses it to see if it’s relevant to your search keywords. By contrast, most university search databases rely only on the titles of articles.

Use the power of the best quality search engine in the world to find scholarly sources .

Note: Google and Google Scholar are different search engines.

To use Google Scholar, go to: https://scholar.google.com

Then, search on google scholar using keywords. I’m going to search keywords for an essay on the topic: “What are the traits of a good nurse?”

If you really like the idea of that first source, I recommend copying the title and trying your University online search database. Your university may give you free access.

4. Cite at least 50% sources you found on your Own Research

Okay, so I’ve told you that you should cite both assigned readings and readings you find from Google Scholar.

Here’s the ideal mix of assigned sources and sources that you found yourself: 50/50.

Your teacher will want to see that you can use both assigned readings and do your own additional research to write a top essay . This shows you’ve got great research skills but also pay attention to what is provided in class.

I recommend that you start with the assigned readings and try to get as much information out of them, then find your own additional sources beyond that using Google Scholar.

So, if your essay has 10 citations, a good mix is 5 assigned readings and 5 readings you found by yourself.

5. Cite Newer Sources

As a general rule, the newer the source the better .

The best rule of thumb that most teachers follow is that you should aim to mostly cite sources from the past 10 years . I usually accept sources from the past 15 years when marking essays.

However, sometimes you have a really great source that’s 20, 30 or 40 years old. You should only cite these sources if they’re what we call ‘seminal texts’. A seminal text is one that was written by an absolute giant in your field and revolutionized the subject.

Here’s some examples of seminal authors whose old articles you would be able to cite despite the fact that they’re old:

• Education: Vygotsky, Friere, Piaget
• Sociology: Weber, Marx, C. Wright Mills
• Psychology: Freud, Rogers, Jung

Even if I cite seminal authors, I always aim for at least 80% of my sources to have been written in the past 10 years.

6. Reference twice per Paragraph

How much should you reference?

Here’s a good strategy: Provide two citations in every paragraph in the body of the essay.

It’s not compulsory to reference in the introduction and conclusion . However, in all the other paragraphs, aim for two citations.

Let’s go over the key strategies for achieving this:

• These two citations should be to different sources, not the same sources twice;
• Two citations per paragraph shows your points are backed up by not one, but two expert sources;
• Place one citation in the first half of the paragraph and one in the second half. This will indicate to your marker that all the points in the whole paragraph are backed up by your citations.

This is a good rule of thumb for you when you’re not sure when and how often to reference. When you get more confident with your referencing, you can mix this up a little.

7. The sum total of your sources should be minimum 1 per 150 words

You can, of course, cite one source more than once throughout the essay. You might cite the same source in the second, fourth and fifth paragraphs. That’s okay.

But, you don’t want your whole essay to be based on a narrow range of sources. You want your marker to see that you have consulted multiple sources to get a wide range of information on the topic. Your marker wants to know that you’ve seen a range of different opinions when coming to your conclusions.

When you get to the end of your essay, check to see how many sources are listed in the end-text reference list. A good rule of thumb is 1 source listed in the reference list per 150 words. Here’s how that breaks down by essay size:

• 1500 word essay: 10 sources (or more) listed in the reference list
• 2000 word essay: 13 sources (or more) listed in the reference list
• 3000 word essay: 20 sources (or more) listed in the reference list
• 5000 word essay: 33 sources (or more) listed in the reference list

8. Instantly improve your Reference List with these Three Tips

Here’s two things you can do to instantly improve your reference list. It takes less than 20 seconds and gives your reference list a strong professional finish:

a) Ensure the font size and style are the same

You will usually find that your whole reference list ends up being in different font sizes and styles. This is because you tend to copy and paste the titles and names in the citations from other sources. If you submit the reference list with font sizes and styles that are not the same as the rest of the essay, the piece looks really unprofessional.

So, quickly highlight the whole reference list and change its font to the same font size and style as the rest of your essay. The screencast at the end of Step 8 walks you through this if you need a hand!

b) List your sources in alphabetical order.

Nearly every referencing style insists that references be listed in alphabetical order. It’s a simple thing to do before submitting and makes the piece look far more professional.

If you’re using Microsoft Word, simply highlight your whole reference list and click the A>Z button in the toolbar. If you can’t see it, you need to be under the ‘home’ tab (circled below):

You’ve probably never heard of a hanging indent. It’s a style where the second line of the reference list is indented further from the left-hand side of the page than the first line. It’s a strategy that’s usually used in reference lists provided in professional publications.

If you use the hanging indent, your reference list will look far more professional.

Here’s a quick video of me doing it for you:

9. Do one special edit especially for Referencing Style

The top students edit their essays three to five times spaced out over a week or more before submitting. One of those edits should be specifically for ensuring your reference list adheres to the referencing style that your teacher requires.

To do this, I recommend you get that cheat sheet printout that I mentioned in Step 1 and have it by your side while you read through the piece. Pay special attention to the use of commas, capital letters, brackets and page numbers for all citations. Also pay attention to the reference list: correct formatting of the reference list can be the difference between getting the top mark in the class and the fifth mark in the class. At the higher end of the marking range, things get competitive and formatting of the reference list counts.

A Quick Summary of the 9 Top Strategies…

Follow the rules of your referencing style guide (and that cheat sheet I recommended!) and use the top 9 tips above to improve your referencing and get top marks. Not only will your referencing look more professional, you’ll probably increase the quality of the content of your piece as well when you follow these tips!

Here’s a final summary of the 9 top tips:

Strategies for How to Reference in an Essay (9 Strategies of Top Students)

• Print out your Reference Style Cheat Sheet
• Only cite Experts
• Always use Google Scholar
• Cite at least 50% sources you found on your Own Research
• Cite Newer Sources
• Reference twice per Paragraph
• The sum total of your sources should be minimum 1 per 150 words
• Instantly improve your Reference List with these Three Tips
• Do one special edit especially for Referencing Style

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 5 Top Tips for Succeeding at University
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 50 Durable Goods Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 100 Consumer Goods Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 30 Globalization Pros and Cons

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

How to Write a Reference List (or Bibliography) For an Essay

An essay without a reference list is like a house without foundations – weak and unsupported!

After all, the reference list is ‘proof’ that the books and journals you referred to in your essay do exist. In turn, this makes your essay seem more credible.

But a reference list will only enhance your essay if it is accurate . That said, let’s explore how to write a clear and accurate reference list for an essay.

How to label your list of references

Firstly, make sure you know what to call the list of references at the end of your essay. The most common name for this list is a ‘reference list’. But some referencing styles call it a ‘bibliography’ or even a ‘works cited’ list. Also, it’s possible to have a ‘reference list’ and a ‘bibliography’ in the same essay.

What’s the difference between a reference list and a bibliography?

Generally speaking, a ‘reference list’ includes a list of all the sources that were cited in the essay – nothing more and nothing less. A bibliography, on the other hand, includes works that were consulted but not specifically cited in the essay. This is the traditional meaning of the term bibliography, at least.

But, in OSCOLA style, the ‘bibliography’ functions more like a ‘reference list’.

Confused? Don’t worry! This table will show you how to label the list of references according to your chosen referencing style.

Referencing styleHow to label the refs at the end of the essayDescriptionAPA

General rules to follow

Once you know how to label your list of references, you can start putting the list together. Here are some general rules that apply to all referencing styles:

• Start your list of references on a new page – it looks a lot neater!
• Get the placement right – references usually come at the end of the essay but before the appendix (if applicable).
• Alphabetical order – the references should be arranged in alphabetical order (by surname).
• Remove hyperlinks – that way, your reference list will look neat and tidy when it’s viewed on-screen.
• Don’t change Americanisms – References should be written in their original form. So, if you’re citing the ‘Journal of Behavior Studies’ , don’t be tempted to change this to the ‘Journal of Behaviour Studies’ .
• Word count – Remember that the reference list does not contribute to the total word count, so remember to deduct these words when you calculate the final word count.

When looking for sources, you might have noticed that some publications offer ‘suggested citations’. It can be helpful to copy and paste these suggested citations, but you will probably need to make some changes to ensure the citation is compliant with your referencing style. That said, let’s take a look at each referencing style in a bit more depth.

How to write a reference list in APA style

Key points to remember:

• As a minimum, the reference should contain the author’s name , the date of the publication, the title , and the source (I.e. where it came from).
• Additional information is also required for journals, such as the page number(s), the volume number and the issue number (see example).
• The doi should be provided at the end of the reference (if applicable).
• All lines except the first line should be indented – this is called a hanging indent. (Word: Paragraph>Special>Hanging).
• Remember to put a full stop at the end of each reference.

For further guidance, check out APA Seventh Edition ! This resource is great as it provides plenty of examples.

How to write a bibliography in OSCOLA

At the end of your essay, you should report a ‘Table of Cases’ a ‘Table of Legislation’, and finally, a ‘Bibliography’. In OSCOLA, the bibliography should include all secondary sources that were cited in the essay.

The secondary sources are listed in a very similar way to the footnotes except that the author’s name is inverted (surname, first initial).

If there are any unattributed works, these should begin with ——.

You’ll notice that OSCOLA is a pretty minimalist referencing style. This means it’s quite easy to get the hang of. You can find full and detailed guidance in this OSCOLA referencing handbook .

How to write a reference list in Harvard style

There is no official manual for Harvard style like there is for APA and Chicago. Rather, universities adopt their own versions of Harvard style. So, if your faculty uses Harvard style, get a hold of your university’s referencing guide to check the requirements.

• Generally speaking, though, a Harvard-style reference list is similar to an APA-style reference list, in that you must provide the author’s name, date of publication, title, and source. Similarly, book titles and journal titles should be italicised.
• Unlike APA, there is no need to add a hanging indent.
• Finally, when citing books, you should provide both the publisher’s name and location (Publisher Location: Publisher Name).

This Harvard referencing guide from The University of East Anglia is comprehensive yet easy to understand – definitely one of the best guides out there!

ASA reference list guidance

• ASA is fairly similar to APA but notice the differences in punctuation (see examples).
• The reference list must be double-spaced.
• You should include the author’s first name and surname (unless the first name was not included in the original publication).
• Also, the first author’s name should be inverted (surname, first name) but any subsequent names should not be inverted (first name, surname).

Note how a colon is used to introduce the page numbers. This is one of the key differences between APA and ASA style. For more information, check out the ASA quick style guide .

MLA ‘works cited’ guidance

As mentioned, the works cited list is equivalent to a reference list, so it must list all the publications that were cited in the essay.

• The references should be formatted with a hanging indent (like APA).
• Uniquely, the date comes towards the end of the reference.
• First and last names are required (inverted)
• The publisher’s name is required but the location is not.

Want to know more? This MLA resource is highly recommended!

How to write a reference list in Chicago style (in-text references)

• Like APA and MLA, the references should be indented (hanging)
• The author’s first name and surname should be provided
• For books, the publisher’s location and name are required.
• Notice that commas are rarely used (except to separate the volume and issue number of a journal).
• A doi should be provided after an electronic resource. If there is no doi, a URL is acceptable.

The Chicago Manual of Style is updated regularly so always use the latest guidance. Finally, if you are using the Chicago footnote style of referencing, check out the bibliography guidance here .

Is the reference list really that important?

In a word, yes!

Imagine, for a moment, that you’ve just finished reading an essay…

Overall, it made some interesting points, but there were no references to back up the claims that were made. Would you think this was a good essay? Would you trust what the author had written? Or would you think it was lacking?

Once you see things from the reader’s perspective, the importance of the reference list suddenly becomes clear.

In essence, this special list boosts the credibility of your essay. So, don’t make it an after-thought.

Need help with your referencing list or bibliography? Our essay writing service can help!

You may also like

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Reference List: Basic Rules

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

This resourse, revised according to the 7 th  edition APA Publication Manual, offers basic guidelines for formatting the reference list at the end of a standard APA research paper. Most sources follow fairly straightforward rules. However, because sources obtained from academic journals  carry special weight in research writing, these sources are subject to special rules . Thus, this page presents basic guidelines for citing academic journals separate from its "ordinary" basic guidelines. This distinction is made clear below.

Note:  Because the information on this page pertains to virtually all citations, we've highlighted one important difference between APA 6 and APA 7 with an underlined note written in red.  For more information, please consult the   Publication Manual of the American Psychological Association , (7 th  ed.).

Formatting a Reference List

Your reference list should appear at the end of your paper. It provides the information necessary for a reader to locate and retrieve any source you cite in the body of the paper. Each source you cite in the paper must appear in your reference list; likewise, each entry in the reference list must be cited in your text.

Your references should begin on a new page separate from the text of the essay; label this page "References" in bold, centered at the top of the page (do NOT underline or use quotation marks for the title). All text should be double-spaced just like the rest of your essay.

Basic Rules for Most Sources

• All lines after the first line of each entry in your reference list should be indented one-half inch from the left margin. This is called hanging indentation.
• All authors' names should be inverted (i.e., last names should be provided first).
• For example, the reference entry for a source written by Jane Marie Smith would begin with "Smith, J. M."
• If a middle name isn't available, just initialize the author's first name: "Smith, J."
• Give the last name and first/middle initials for all authors of a particular work up to and including 20 authors ( this is a new rule, as APA 6 only required the first six authors ). Separate each author’s initials from the next author in the list with a comma. Use an ampersand (&) before the last author’s name. If there are 21 or more authors, use an ellipsis (but no ampersand) after the 19th author, and then add the final author’s name.
• Reference list entries should be alphabetized by the last name of the first author of each work.
• For multiple articles by the same author, or authors listed in the same order, list the entries in chronological order, from earliest to most recent.
• Note again that the titles of academic journals are subject to special rules. See section below.
• Italicize titles of longer works (e.g., books, edited collections, names of newspapers, and so on).
• Do not italicize, underline, or put quotes around the titles of shorter works such as chapters in books or essays in edited collections.

Basic Rules for Articles in Academic Journals

• Present journal titles in full.
• Italicize journal titles.
• For example, you should use  PhiloSOPHIA  instead of  Philosophia,  or  Past & Present   instead of  Past and Present.
• This distinction is based on the type of source being cited. Academic journal titles have all major words capitalized, while other sources' titles do not.
• Capitalize   the first word of the titles and subtitles of   journal articles , as well as the   first word after a colon or a dash in the title, and   any proper nouns .
• Do not italicize or underline the article title.
• Deep blue: The mysteries of the Marianas Trench.
• Oceanographic Study: A Peer-Reviewed Publication

Please note:  While the APA manual provides examples of how to cite common types of sources, it does not cover all conceivable sources. If you must cite a source that APA does not address, the APA suggests finding an example that is similar to your source and using that format. For more information, see page 282 of the   Publication Manual of the American Psychological Association , 7 th  ed.

Writing Better University Essays/Referencing

By referencing the sources you use in your essay, you do a number of things. First of all, you comply with an academic convention. Secondly, you make your essay look more professional. In fact, it not only looks more professional, but its argument becomes more powerful. Thirdly, you allow others to check your sources. This is often only a hypothetical issue, but a look through the list of your references will allow others to judge your argument quickly. Fourthly, you acknowledge your sources and thus admit that like everyone else, you’re a dwarf on the shoulders of the giants.

The essential bits of referencing require you to provide enough information to others so that they can identify the source. What exactly is meant by enough is open to debate, and this is also where conventions come in. Essential is that you do provide references. Ideally, you would do so properly. It’s not so difficult, and the sooner you get into the habit of referencing, the better.

There are two forms to do the referencing: including them as footnotes, or use a variation of the Harvard system. Your institution may have a preference, or even a house style. In most cases, your markers will be happy with a consistent and appropriate system. The Harvard system is also known as author/date, and will be described here in more detail.

• 1 Inside the Text
• 2 At the End
• 3 Problem Cases
• 4 Plagiarism
• 5 Citations and Quotations
• 6 When to Put the References

Inside the Text [ edit | edit source ]

Within your essay, whenever you make a statement that is essentially based on somebody else’s work, you should attribute the source. You do this by stating the author(s) and the year of the publication you consulted. Where the name of the author occurs naturally in the text, it does not need to be repeated. The references are usually included at the end of a sentence, or where inappropriate in a place where the text flow is not interrupted too much, such as in front of a comma. This may be necessary, for example, if only the first half of your sentence is based on someone else’s work.

The name of the author is included in brackets, together with the year of publication. Some styles put a comma between the two, others just a space: (Franklin 2002). Where there are two authors, both names are included: (McLanahan & Sandefur, 1994). Some styles prefer the word and , others prefer the ampersand (& symbol). Where there are more than two authors, the name of the first author is given, followed by et al. (which literally means and others ): (Almeder et al. , 2001). Some styles put et al. into italics, others don’t.

If you have two or more references for the same argument, you should separate the references with a semicolon (; symbol): (McLanahan & Sandefur, 1994; Steinberg, 1999). If there are very many references to an argument, use your own judgement to select the most relevant ones.

What should you reference? Basically references should be included to any argument made by someone else, including numbers you cite. However, statements of general nature need not be attributed to anyone. A statement that the sky is blue alone does not require a reference. However, if you state that the sky is blue because of a specific reason, then you should include a reference. If you use the exact words of an author (quotation), you’ll need to give the number of the page where you copy from. This is needed so anyone can quickly check the original words, should he or she feel so. See the separate section on quotes.

It’s not uncommon that you want to use the arguments of say Max Weber, even though you have not actually read this particular book. Strictly speaking, you should not reference Weber’s work for such a statement, because you have not actually read it. Can you really be sure this is what Weber said or meant? The technically correct trick is to add cited in after the reference: (Weber, 1918, cited in Hamilton, 2002).

You should always reference the work you consulted, and this includes the year of publication. Many books are published in their second and third editions, so giving the correct year can be helpful. Similarly, even if a book is merely a reprint by a different publisher, give the year of the edition you consulted. The page numbers may differ. If it’s just a second print of the exact same book, use the original date. Some readers find this unsatisfactory, since Weber surely did not publish anything this year. The convention to circumvent this issue is to give both years: the year of the original publication, together with the one of the work you consulted. Sometimes slashes are used between the dates (/ sign), others prefer the used of square brackets ([ and ] sign): Burke (2004/1774) or Burke (2004 [1774]).

Another small issue occurs where an author published more than one book or article in a single year, and you want to cite more than one of them. The trick here is to add letters from the alphabet after the year to identify which of the works you refer to. Use the letter a for the first of your references, the letter b for the second and so on: (McManus, 1994a) and (McManus, 1994b) are two different works.

To sum it up, inside the text, you give the family name of the author, followed by the year of the publication. Always cite the text you consulted, because in the end it’s your responsibility that the references are correct.

At the End [ edit | edit source ]

At the end of your essay you should include a list of references. Such a list of references provides more details than just the name of the author and the year of publication. It’s this list that allows identifying the work cited. Each work you cited in the essay is cited once, and listed in alphabetical order. Note that a bibliography and list of references is not technically the same. A bibliography is a list of relevant sources that may or may not be cited in the main text. References are the sources you cited, even if they are rather trivial. Use the heading references for your references.

For books, you put the family name of the author(s) and their initials, followed by the year of publication in brackets, the title in italics, the place of publication, and finally the name of the publisher. If there are editors, give their names instead of the authors’. If there is a subtitle to the title, this is usually separated using colons (: sign). Where there are more than four authors, it’s common to use et al. after the first three, but some styles insist on citing all authors. Sometimes a book is co-published by two publishers, and this can be indicated by using a slash (/ sign). Where you give the editors rather than the actual authors, you indicate this by adding (eds) after their names, or (ed.) if there is only one. The title is capitalized. For example:

Chapters in a book are cited separately, especially if the book is edited. You give the family name of the author and his or her initial, the year, the name of the chapter in single speech marks (‘ and ’ sign; not capitalized), followed by the word in , and the name and year of the editor(s). If you cite only one chapter, you can give the whole reference at the end; otherwise it’s enough to give the name and year of the editor. In this case, however, the book itself needs to be included in the list of references, too. For example:

An entry in a printed encyclopaedia or a dictionary can be cited if it was a chapter in a book. The editors are often given on the front of the reference book. For example:

Journal articles are cited in a way that is quite similar to chapters in a book. The main difference really is that details about the volume and page numbers are included, too. The reference starts with the name and initial of the author, the year in brackets, the title of the article in single speech marks (not capitalized), followed by the name of the journal in italics (capitalized), and further details. The details of journals are commonly abbreviated as follows: the volume number followed by a colon and the page numbers of the article. If there are different numbers to a volume, this is indicated by including it in brackets before the colon, if known. Online journals may not have page numbers. For example:

Pages on the internet should be cited where used. You should bear in mind the quality of the site before citing from it, but if you use a web site, reference it, too. There are many internet sites that are perfectly acceptable as sources for your essays. The reference includes the name of the author and initial, the year in brackets, the title of the document in italics, the word online in square brackets, the place of publication, the publisher, the words available from : followed by the URL, and the date when the document was accessed in brackets. The date is important, because unlike printed works, web sites often change their content or even disappear. Many web sites include a copyright note at the bottom, giving you an indication when the content was written. For example:

Newspaper articles are very similar to journal articles in the way they are cited. The key difference is that rather than the volume, the date is given. The reference therefore includes the name and initial of the author, the year of publication in brackets, the title in single speech marks, the name of the newspaper in italics (capitalized), the date, and finally the page where the article was found. For one page it’s customary to use the abbreviation p. , for articles running over two or more pages, the abbreviation pp. is common. For example:

Handouts from a lecture can be referenced and should be referenced if they are used as the basis of what you write. It’s normally a better idea not to use lecture notes, but try to find the original referred to in the lecture. Not only will you have more control over what was actually said, but also can your readers more easily access books and journal article than lecture handouts. The reference to a lecture handout includes the name and initial of the lecturer, the year in bracket, the title of the handout in single speech marks, the words lecture notes distributed in followed by the name of the course in italics, the word at and the name of your institution, the place, and date of the lecture. For example:

Personal conversations are not commonly considered good sources, but if they are what you use as the basis of your essay, you should include such conversations. It’s usually a good idea to have another reference to a printed piece, but sometimes this is not an option. In terms of giving the reference, personal conversations are very easy: the name of the person you spoke to, the year in brackets, the words conversation with the author and the date of the conversation. For example:

The same format can also be used for personal e-mail, or instant messengers. Once again, bear in mind the credibility of your sources. With e-mail messages it’s customary to include the e-mail address of the sender in brackets after the name, but it’s essential that you obtain consent from the author. The subject line of the e-mail is often included as the title. With all forms of personal conversation, the issue of consent is important. It’s always a very good idea to check with the author first.

Problem Cases [ edit | edit source ]

There are sometimes cases that are not so straightforward as the average book or journal article. For everything there is a solution in the academic conventions. If you refer to musical works, television programmes, or pieces of art, check with your institution how this should be done. If everything else fails, remember the function of referencing, and provide a reasonable amount of information for others to chase the work. Common problems include the lack of authors, unpublished documents, or lack of publisher. Where there is no author, often there is an organization. Put the name of the organization. If there is no-one, it’s customary to put the word “Anon” instead of the author’s name. For example:

Sometimes the year of a document is not known. Where you have a rough idea, you can put a c before the date, such as in (c.1999). Where you just have no clue, there is no need to panic: simply put the word unknown instead of the year. Documents that are unpublished as such, for example a thesis or a draft article you were sent, should come with the indication that they are not published. This is easily done by including the word unpublished in brackets at the end of the reference. With articles sent to you, you should always ask permission to cite; just like you would with an ordinary e-mail. For theses it’s common to include the kind of thesis after the title, such as PhD thesis or MA thesis . Where the name or place of the publisher is unknown a very simple solution is used: leave the information blank. This is particularly an issue with internet sites. Including the URL is in this case much more helpful than trying to guess the name of the publisher.

Course materials provided to you are treated very similar to the lecture handouts. Give the name of the author, the year in brackets, the course code if there is one, the course title in italics (capitalized), the kind of material and its title in single speech marks, place of publication, and publisher. For example:

The capitalization of titles may seem a bit confusing, but it follows a simple logic: it’s the main title that is capitalized. In the case of a book, the main title is that of the book. In the case of journal articles, on the other hand, the main title is thought to be that of the journal itself. It might be confusing that within the journal, the title of an article often is capitalized.

Capitalization is not very hard to achieve. Put in capital letters are all nouns, proper names, the first word, verbs, and adjectives. This is in fact almost everything. Not put in capital letters are words like and , in , or , or with . Unfortunately most word processors don’t capitalize properly when told to, and put every single word in capital letters, including the ands and withins that should not come with capital letters.

Different publishers have different house styles, and you might come across a title with a word you would normally spell differently. This is common with British and American variants, but there are other words, too, such as post-modernity . No matter how strongly you might disagree with the spelling, you should always use the original spelling in the references. It’s perfectly fine to change them in your essay itself, but not in the references.

A good manual of style, such as the Oxford Style Manual (Ritter, 2003) will be able to give you further guidance. Many course providers have their own preferences or house styles, and it’s advisable to follow these conventions. Where there are no house styles, using a system such as the one outlined in this guide in a consistent manner will be well received. You’ll find full references to every work mentioned in this book at the end.

Plagiarism [ edit | edit source ]

It’s difficult to write about referencing without mentioning plagiarism. Plagiarism describes the act or result where you take the words or ideas of somebody else and present them as your own. Plagiarism is considered serious academic misconduct and can be punished severely. Most importantly, however, your reputation is on the line.

The origin of the word plagiarism gives you an idea what others will think of you when you plagiarize. The word goes back to the Latin plagiārius , a thief and kidnapper—in particular a child snatcher and somebody abducting slaves. The modern use in academia brands you a literary thief (OED, 2005).

There are a number of reasons why plagiarism occurs. The worst case is deliberate plagiarism (for whatever reason). Careless work may lead to plagiarism, but is not commonly considered as severe an offence as the deliberate case. Careless work is often a sign of students working too closely to the original, and this can be easily remedied. Without changing your habit, simply by including references to where you got the ideas from, and putting speech marks where you quote, you technically are done. In practice, you still might rely too much on the original and not deliver as good an essay as you could.

Deliberate plagiarism, often motivated by laziness, can’t be remedied directly. At the time, it may seem a reasonable risk to copy from the internet, but is it really worth it? Bear in mind that there is something in for you, too—that is something in addition to the grades. The more you write, the easier it gets.

If you work too closely to the original, there is a simple solution: don’t write the essay with the books in front of you. By so doing, there is very little danger that you copy word by word. In a way, you force yourself to make the material your own: and that is a good thing—it makes a better argument, your essay will be more original, and not least, you’ll also get better grades. Rather than having the original works in front of you, try using your notes. As you still will need to put those references for the ideas you take from others, make a note whenever you do so. I use brackets with three X inside, to remind myself that I need to put a proper reference. Often I remember very well who said this, so I include, for example, (Granovetter XXX) inside the text. When checking the essay, it’s hard not to notice the triple X; and there is always the search facility in the word processor. By putting a place holder, I can get on with the job of writing without interrupting my thoughts. Equally important, I leave some traces indicating to myself that there is some more work to be done: finding the proper reference, for example.

If you think plagiarism is hard to detect by your marker, think again. There are a great number of signs that give plagiarized work away. Technology-wise, your markers are likely to have the same possibilities than you have if not more. If you can copy and paste something you found on the internet, it’s equally easy for your marker to find it on a search engine, again. It would, of course, be possible, to change plagiarized work to the extent that the deed is no longer easy to spot. Usually, however, this is just as much work as writing the essay yourself.

Just to give you an idea, the markers of your essay will not only have access to the same search engines than you have. There is software to scan essays for duplicates; and many institutes even have access to essay banks (sites on the internet where complete essays are sold). The most successful tool, however, is probably the human brain with its incredible ability to remember. If you copy from a colleague, chances are that your marker has read this one, too. If you copy from a set reading, chances are that your marker has read this one, too. Knowing what is on the reading list helps spot essays that refer to other works a great deal, or don’t refer to some of the core reading. Your marker can estimate how many readings you had time to read, or whether you’re likely to have read a great number of papers on the Belgian perspective of whatever issues is set in the question. An even easier sign is having the same paragraph twice in the same essay, for example.

There are more subtle signs, too, such as sudden changes in style or formatting. Many people are unaware of how idiosyncratic one’s writing style is. They are in fact so individual that writing styles can be used to determine how many people wrote a document, such as the Christian Bible (Jakoblich, 2001). Writing style includes the tenses we use, the level of formality, our own choice of words, the kinds of metaphors we put, whether we use American or British English, choices over punctuation, the length of sentences, or the use of specialist terms. Typographic signs include font size, choices of where to break paragraphs, spaces in between lines, and things like proper m- and n-dashes (when copying from electronic articles).

The presence or lack of references is often an easy sign: for example, where there are many references inside the text, but few at the end, or where the citation style changes within a single essay. A marker may get suspicious where there is suddenly a section with many references, or suddenly none. Sometimes, students even include hyperlinks in references when copying from electronic journals; and have them automatically underlined by the word processor.

Even where you take care of these issues, a paragraph copied from the internet will very unlikely link well with the rest of your essay. The style may be inappropriate, or just different. Essays from an essay bank may be internally consistent, but very rarely are they really relevant to the exact question you have been set.

In summary, you can avoid plagiarism easily. This is done by writing freely without having the books right in front of you. Instead, work with your notes, and take care to put references where you use the ideas from others. Don’t use the internet to copy from, no matter how tempting it is. It will hardly ever be worth it.

Citations and Quotations [ edit | edit source ]

There is an important difference between citations and quotations. Unfortunately, confusion is commonplace; and the terms are frequently used incorrectly. Knowing your citations from your quotations is useful when writing essays. It’s essential, in fact, if you want to reference properly.

Citations are about ideas you take from others. Quotations are about the exact words used by others. This is really the whole distinction. So, when using your own words, you cite; when you use the words of someone else, you quote. “Why can’t a man be more like a woman?” (Blankenhorn, 1995, p.117) is a quotation, because I use the exact same words Blankenhorn did. However, when stating that families in the US are increasingly defined by the absence of a father (Blankenhorn, 1995), I only use the idea, not the exact words.

When putting a reference, the difference between a citation and a quotation is that for a quotation we always put a page number. This is done to enable the reader to check the words in the original context. In the list of references at the end of the text, there is no difference.

Short quotations are included in the text, and enclosed by speech marks. Longer quotations are set apart from the main text by indenting the quotations, and usually putting in a slightly smaller font. Longer means about 3 to 4 lines or more. For example:

When quoting someone else, you should take great care to copy the words exactly. Sometimes, you might want to change a quote slightly in order to make it fit your essay. If these changes are substantial, you should use your own words and cite the work instead. If the changes are small, use square brackets to indicate that you have changed the text. For example, you might quote Rawls (1999, p.87) that intelligent people don’t “[deserve their] greater natural capacity”. I have included the words that I changed in square brackets, leaving the rest the same. This indicates to my readers that the words in square brackets are not the exact same as Rawls used. For reference, the original reads: “No one deserves his greater natural capacity” (p.87). I made the changes, because I wrote about intelligent people, and Rawls was talking in more general terms.

Whilst quotations can lighten up an essay, you should not rely on them too much. Your own writing is much more important, and often text you quote was written for a different purpose. The consequence is that the quotations may be relevant in content (what is being said), but in terms of style don’t fit well with what you wrote. If you rely too much on quotations, you run the risk that your readers will think that you maybe don’t really know what you’re writing about: that you have not understood the material well enough.

When to Put the References [ edit | edit source ]

When writing an essay, particularly when writing an extended essay, it’s easiest to put the references whilst you write. This is the case, because you still know where you got the idea from. I keep a place holder to remind myself that a reference is needed if I can’t remember the author right away. Often, I will know at least some of it, and write this down. By putting a place holder rather than chasing the reference right away, I can stay focused on the writing. However, I also indicate that the essay is not completed. Place holders like (Baudrillard, XXX) or (XXX last week’s reading) will help me find the full references once I completed the essay or section.

References are needed whenever you write an academic piece of writing. Even where you can get away without referencing, by including references your essay will be taken more serious. It’s a good habit to put references all the time, so when you really need to—such as in your thesis—you’ll not struggle, or spend days trying to find out how to reference a chapter in a book.

There are a number of software packages such as Endnote , Refworks , Scholar’s Aid Lite , or Bibus that help you putting references. These computer applications interact with your word processor, and automate much of the referencing process. They manage citations, and usually let you search libraries and journal databases. Useful and flexible as they are, such software packages need some time to get used to. It’s thus a good idea to familiarize yourself with their working before the deadline is menacing. For example, make sure you know how to put page numbers for quotations.

Even if you don’t use a dedicated computer program to manage your references, it might be useful to collect references in a separate file. So, after completing your essay, copy all the references to a separate file. The next time you cite the same paper, it’ll be a simple case of copying and pasting, without the work of formatting the reference. Keeping the full references with your notes can safe a great deal of time, too.

Next: Exam essays

• Book:Writing Better University Essays

Navigation menu

• Walden University
• Faculty Portal

Reference List: Common Reference List Examples

Article (with doi).

Alvarez, E., & Tippins, S. (2019). Socialization agents that Puerto Rican college students use to make financial decisions. Journal of Social Change , 11 (1), 75–85. https://doi.org/10.5590/JOSC.2019.11.1.07

Laplante, J. P., & Nolin, C. (2014). Consultas and socially responsible investing in Guatemala: A case study examining Maya perspectives on the Indigenous right to free, prior, and informed consent. Society & Natural Resources , 27 , 231–248. https://doi.org/10.1080/08941920.2013.861554

Use the DOI number for the source whenever one is available. DOI stands for "digital object identifier," a number specific to the article that can help others locate the source. In APA 7, format the DOI as a web address. Active hyperlinks for DOIs and URLs should be used for documents meant for screen reading. Present these hyperlinks in blue and underlined text (the default formatting in Microsoft Word), although plain black text is also acceptable. Be consistent in your formatting choice for DOIs and URLs throughout your reference list. Also see our Quick Answer FAQ, "Can I use the DOI format provided by library databases?"

Jerrentrup, A., Mueller, T., Glowalla, U., Herder, M., Henrichs, N., Neubauer, A., & Schaefer, J. R. (2018). Teaching medicine with the help of “Dr. House.” PLoS ONE , 13 (3), Article e0193972. https://doi.org/10.1371/journal.pone.0193972

For journal articles that are assigned article numbers rather than page ranges, include the article number in place of the page range.

For more on citing electronic resources, see  Electronic Sources References .

Article (Without DOI)

Found in a common academic research database or in print.

Casler , T. (2020). Improving the graduate nursing experience through support on a social media platform. MEDSURG Nursing , 29 (2), 83–87.

If an article does not have a DOI and you retrieved it from a common academic research database through the university library, there is no need to include any additional electronic retrieval information. The reference list entry looks like the entry for a print copy of the article. (This format differs from APA 6 guidelines that recommended including the URL of a journal's homepage when the DOI was not available.) Note that APA 7 has additional guidance on reference list entries for articles found only in specific databases or archives such as Cochrane Database of Systematic Reviews, UpToDate, ProQuest Dissertations and Theses Global, and university archives. See APA 7, Section 9.30 for more information.

Found on an Open Access Website

Eaton, T. V., & Akers, M. D. (2007). Whistleblowing and good governance. CPA Journal , 77 (6), 66–71. http://archives.cpajournal.com/2007/607/essentials/p58.htm

Provide the direct web address/URL to a journal article found on the open web, often on an open access journal's website. In APA 7, active hyperlinks for DOIs and URLs should be used for documents meant for screen reading. Present these hyperlinks in blue and underlined text (the default formatting in Microsoft Word), although plain black text is also acceptable. Be consistent in your formatting choice for DOIs and URLs throughout your reference list.

Weinstein, J. A. (2010).  Social change  (3rd ed.). Rowman & Littlefield.

If the book has an edition number, include it in parentheses after the title of the book. If the book does not list any edition information, do not include an edition number. The edition number is not italicized.

American Nurses Association. (2015). Nursing: Scope and standards of practice (3rd ed.).

If the author and publisher are the same, only include the author in its regular place and omit the publisher.

Lencioni, P. (2012). The advantage: Why organizational health trumps everything else in business . Jossey-Bass. https://amzn.to/343XPSJ

As a change from APA 6 to APA 7, it is no longer necessary to include the ebook format in the title. However, if you listened to an audiobook and the content differs from the text version (e.g., abridged content) or your discussion highlights elements of the audiobook (e.g., narrator's performance), then note that it is an audiobook in the title element in brackets. For ebooks and online audiobooks, also include the DOI number (if available) or nondatabase URL but leave out the electronic retrieval element if the ebook was found in a common academic research database, as with journal articles. APA 7 allows for the shortening of long DOIs and URLs, as shown in this example. See APA 7, Section 9.36 for more information.

Chapter in an Edited Book

Poe, M. (2017). Reframing race in teaching writing across the curriculum. In F. Condon & V. A. Young (Eds.), Performing antiracist pedagogy in rhetoric, writing, and communication (pp. 87–105). University Press of Colorado.

Include the page numbers of the chapter in parentheses after the book title.

Christensen, L. (2001). For my people: Celebrating community through poetry. In B. Bigelow, B. Harvey, S. Karp, & L. Miller (Eds.), Rethinking our classrooms: Teaching for equity and justice (Vol. 2, pp. 16–17). Rethinking Schools.

Also include the volume number or edition number in the parenthetical information after the book title when relevant.

Freud, S. (1961). The ego and the id. In J. Strachey (Ed.),  The standard edition of the complete psychological works of Sigmund Freud  (Vol. 19, pp. 3-66). Hogarth Press. (Original work published 1923)

When a text has been republished as part of an anthology collection, after the author’s name include the date of the version that was read. At the end of the entry, place the date of the original publication inside parenthesis along with the note “original work published.” For in-text citations of republished work, use both dates in the parenthetical citation, original date first with a slash separating the years, as in this example: Freud (1923/1961). For more information on reprinted or republished works, see APA 7, Sections 9.40-9.41.

Classroom Resources

Citing classroom resources.

If you need to cite content found in your online classroom, use the author (if there is one listed), the year of publication (if available), the title of the document, and the main URL of Walden classrooms. For example, you are citing study notes titled "Health Effects of Exposure to Forest Fires," but you do not know the author's name, your reference entry will look like this:

Health effects of exposure to forest fires [Lecture notes]. (2005). Walden University Canvas. https://waldenu.instructure.com

If you do know the author of the document, your reference will look like this:

Smith, A. (2005). Health effects of exposure to forest fires [PowerPoint slides]. Walden University Canvas. https://waldenu.instructure.com  

A few notes on citing course materials:

• [Lecture notes]
• [Course handout]
• [Study notes]
• It can be difficult to determine authorship of classroom documents. If an author is listed on the document, use that. If the resource is clearly a product of Walden (such as the course-based videos), use Walden University as the author. If you are unsure or if no author is indicated, place the title in the author spot, as above.
• If you cannot determine a date of publication, you can use n.d. (for "no date") in place of the year.

Note:  The web location for Walden course materials is not directly retrievable without a password, and therefore, following APA guidelines, use the main URL for the class sites: https://class.waldenu.edu.

Citing Tempo Classroom Resources

Clear author: 

Smith, A. (2005). Health effects of exposure to forest fires [PowerPoint slides]. Walden University Brightspace. https://mytempo.waldenu.edu

Unclear author:

Health effects of exposure to forest fires [Lecture notes]. (2005). Walden University Brightspace. https://mytempo.waldenu.edu

Conference Sessions and Presentations

Feinman, Y. (2018, July 27). Alternative to proctoring in introductory statistics community college courses [Poster presentation]. Walden University Research Symposium, Minneapolis, MN, United States. https://scholarworks.waldenu.edu/symposium2018/23/

Torgerson, K., Parrill, J., & Haas, A. (2019, April 5-9). Tutoring strategies for online students [Conference session]. The Higher Learning Commission Annual Conference, Chicago, IL, United States. http://onlinewritingcenters.org/scholarship/torgerson-parrill-haas-2019/

Dictionary Entry

Merriam-Webster. (n.d.). Leadership. In Merriam-Webster.com dictionary . Retrieved May 28, 2020, from https://www.merriam-webster.com/dictionary/leadership

When constructing a reference for an entry in a dictionary or other reference work that has no byline (i.e., no named individual authors), use the name of the group—the institution, company, or organization—as author (e.g., Merriam Webster, American Psychological Association, etc.). The name of the entry goes in the title position, followed by "In" and the italicized name of the reference work (e.g., Merriam-Webster.com dictionary , APA dictionary of psychology ). In this instance, APA 7 recommends including a retrieval date as well for this online source since the contents of the page change over time. End the reference entry with the specific URL for the defined word.

Discussion Board Post

Osborne, C. S. (2010, June 29). Re: Environmental responsibility [Discussion post]. Walden University Canvas.  https://waldenu.instructure.com  

Dissertations or Theses

Retrieved From a Database

Nalumango, K. (2019). Perceptions about the asylum-seeking process in the United States after 9/11 (Publication No. 13879844) [Doctoral dissertation, Walden University]. ProQuest Dissertations and Theses.

Retrieved From an Institutional or Personal Website

Evener. J. (2018). Organizational learning in libraries at for-profit colleges and universities [Doctoral dissertation, Walden University]. ScholarWorks. https://scholarworks.waldenu.edu/cgi/viewcontent.cgi?article=6606&context=dissertations

Unpublished Dissertation or Thesis

Kirwan, J. G. (2005). An experimental study of the effects of small-group, face-to-face facilitated dialogues on the development of self-actualization levels: A movement towards fully functional persons [Unpublished doctoral dissertation]. Saybrook Graduate School and Research Center.

For further examples and information, see APA 7, Section 10.6.

Legal Material

For legal references, APA follows the recommendations of The Bluebook: A Uniform System of Citation , so if you have any questions beyond the examples provided in APA, seek out that resource as well.

Court Decisions

Reference format:

Name v. Name, Volume Reporter Page (Court Date). URL

Sample reference entry:

Brown v. Board of Education, 347 U.S. 483 (1954). https://www.oyez.org/cases/1940-1955/347us483

Sample citation:

In Brown v. Board of Education (1954), the Supreme Court ruled racial segregation in schools unconstitutional.

Note: Italicize the case name when it appears in the text of your paper.

Name of Act, Title Source § Section Number (Year). URL

Sample reference entry for a federal statute:

Individuals With Disabilities Education Act, 20 U.S.C. § 1400 et seq. (2004). https://www.congress.gov/108/plaws/publ446/PLAW-108publ446.pdf

Sample reference entry for a state statute:

Minnesota Nurse Practice Act, Minn. Stat. §§ 148.171 et seq. (2019). https://www.revisor.mn.gov/statutes/cite/148.171

Sample citation: Minnesota nurses must maintain current registration in order to practice (Minnesota Nurse Practice Act, 2010).

Note: The § symbol stands for "section." Use §§ for sections (plural). To find this symbol in Microsoft Word, go to "Insert" and click on Symbol." Look in the Latin 1-Supplement subset. Note: U.S.C. stands for "United States Code." Note: The Latin abbreviation " et seq. " means "and what follows" and is used when the act includes the cited section and ones that follow. Note: List the chapter first followed by the section or range of sections.

Unenacted Bills and Resolutions

(Those that did not pass and become law)

Title [if there is one], bill or resolution number, xxx Cong. (year). URL

Sample reference entry for Senate bill:

Anti-Phishing Act, S. 472, 109th Cong. (2005). https://www.congress.gov/bill/109th-congress/senate-bill/472

Sample reference entry for House of Representatives resolution:

Anti-Phishing Act, H.R. 1099, 109th Cong. (2005). https://www.congress.gov/bill/109th-congress/house-bill/1099

The Anti-Phishing Act (2005) proposed up to 5 years prison time for people running Internet scams.

These are the three legal areas you may be most apt to cite in your scholarly work. For more examples and explanation, see APA 7, Chapter 11.

Magazine Article

Clay, R. (2008, June). Science vs. ideology: Psychologists fight back about the misuse of research. Monitor on Psychology , 39 (6). https://www.apa.org/monitor/2008/06/ideology

Note that for citations, include only the year: Clay (2008). For magazine articles retrieved from a common academic research database, leave out the URL. For magazine articles from an online news website that is not an online version of a print magazine, follow the format for a webpage reference list entry.

Newspaper Article (Retrieved Online)

Baker, A. (2014, May 7). Connecticut students show gains in national tests. New York Times . http://www.nytimes.com/2014/05/08/nyregion/national-assessment-of-educational-progress-results-in-Connecticut-and-New-Jersey.html

Include the full date in the format Year, Month Day. Do not include a retrieval date for periodical sources found on websites. Note that for citations, include only the year: Baker (2014). For newspaper articles retrieved from a common academic research database, leave out the URL. For newspaper articles from an online news website that is not an online version of a print newspaper, follow the format for a webpage reference list entry.

Online Video/Webcast

Walden University. (2013).  An overview of learning  [Video]. Walden University Canvas.  https://waldenu.instructure.com  

Use this format for online videos such as Walden videos in classrooms. Most of our classroom videos are produced by Walden University, which will be listed as the author in your reference and citation. Note: Some examples of audiovisual materials in the APA manual show the word “Producer” in parentheses after the producer/author area. In consultation with the editors of the APA manual, we have determined that parenthetical is not necessary for the videos in our courses. The manual itself is unclear on the matter, however, so either approach should be accepted. Note that the speaker in the video does not appear in the reference list entry, but you may want to mention that person in your text. For instance, if you are viewing a video where Tobias Ball is the speaker, you might write the following: Tobias Ball stated that APA guidelines ensure a consistent presentation of information in student papers (Walden University, 2013). For more information on citing the speaker in a video, see our page on Common Citation Errors .

Taylor, R. [taylorphd07]. (2014, February 27). Scales of measurement [Video]. YouTube. https://www.youtube.com/watch?v=PDsMUlexaMY

Walden University Academic Skills Center. (2020, April 15). One-way ANCOVA: Introduction [Video]. YouTube. https://youtu.be/_XnNDQ5CNW8

For videos from streaming sites, use the person or organization who uploaded the video in the author space to ensure retrievability, whether or not that person is the speaker in the video. A username can be provided in square brackets. As a change from APA 6 to APA 7, include the publisher after the title, and do not use "Retrieved from" before the URL. See APA 7, Section 10.12 for more information and examples.

See also reference list entry formats for TED Talks .

Technical and Research Reports

Edwards, C. (2015). Lighting levels for isolated intersections: Leading to safety improvements (Report No. MnDOT 2015-05). Center for Transportation Studies. http://www.cts.umn.edu/Publications/ResearchReports/reportdetail.html?id=2402

Technical and research reports by governmental agencies and other research institutions usually follow a different publication process than scholarly, peer-reviewed journals. However, they present original research and are often useful for research papers. Sometimes, researchers refer to these types of reports as gray literature , and white papers are a type of this literature. See APA 7, Section 10.4 for more information.

Reference list entires for TED Talks follow the usual guidelines for multimedia content found online. There are two common places to find TED talks online, with slightly different reference list entry formats for each.

TED Talk on the TED website

If you find the TED Talk on the TED website, follow the format for an online video on an organizational website:

Owusu-Kesse, K. (2020, June). 5 needs that any COVID-19 response should meet [Video]. TED Conferences. https://www.ted.com/talks/kwame_owusu_kesse_5_needs_that_any_covid_19_response_should_meet

The speaker is the author in the reference list entry if the video is posted on the TED website. For citations, use the speaker's surname.

TED Talk on YouTube

If you find the TED Talk on YouTube or another streaming video website, follow the usual format for streaming video sites:

TED. (2021, February 5). The shadow pandemic of domestic violence during COVID-19 | Kemi DaSilvalbru [Video]. YouTube. https://www.youtube.com/watch?v=PGdID_ICFII

TED is the author in the reference list entry if the video is posted on YouTube since it is the channel on which the video is posted. For citations, use TED as the author.

Walden University Course Catalog

To include the Walden course catalog in your reference list, use this format:

Walden University. (2020). 2019-2020 Walden University catalog . https://catalog.waldenu.edu/index.php

If you cite from a specific portion of the catalog in your paper, indicate the appropriate section and paragraph number in your text:

...which reflects the commitment to social change expressed in Walden University's mission statement (Walden University, 2020, Vision, Mission, and Goals section, para. 2).

And in the reference list:

Walden University. (2020). Vision, mission, and goals. In 2019-2020 Walden University catalog. https://catalog.waldenu.edu/content.php?catoid=172&navoid=59420&hl=vision&returnto=search

Vartan, S. (2018, January 30). Why vacations matter for your health . CNN. https://www.cnn.com/travel/article/why-vacations-matter/index.html

For webpages on the open web, include the author, date, webpage title, organization/site name, and URL. (There is a slight variation for online versions of print newspapers or magazines. For those sources, follow the models in the previous sections of this page.)

American Federation of Teachers. (n.d.). Community schools . http://www.aft.org/issues/schoolreform/commschools/index.cfm

If there is no specified author, then use the organization’s name as the author. In such a case, there is no need to repeat the organization's name after the title.

In APA 7, active hyperlinks for DOIs and URLs should be used for documents meant for screen reading. Present these hyperlinks in blue and underlined text (the default formatting in Microsoft Word), although plain black text is also acceptable. Be consistent in your formatting choice for DOIs and URLs throughout your reference list.

Related Resources

Knowledge Check: Common Reference List Examples

Didn't find what you need? Search our website or email us .

Read our website accessibility and accommodation statement .

• Previous Page: Reference List: Overview
• Next Page: Common Military Reference List Examples
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Understanding How, When and Why to Reference

Learn how to acknowledge your sources of information

It is important that you acknowledge your sources of information in your academic writing. This allows you to clearly show how the ideas of others have influenced your own work. You should provide a citation (and matching reference) in your essay every time you use words, ideas or information from other sources.  If you would like to learn how, when and why to reference by watching a video, you can do so on Capstone Editing's YouTube channel .

Why reference?

Not referencing correctly can be perceived as plagiarism. It is expected and required at the university level that all your assignments will contain references. Otherwise, you are saying that the essay is made up entirely of your own original ideas, and that you have not engaged critically in any way with the literature. A passing grade requires that you use a minimum number of references (check your assignment marking criteria or ask your lecturer), and a good grade requires many more references than this. The purpose of referencing is to demonstrate the depth and breadth of your research, to show that you have read and engaged with the ideas of experts in your field. It also allows you to give credit to the writers from whom you have borrowed words or ideas. For your reader, referencing allows them to trace the sources of information you have used and to verify the validity of your work. For this reason, your referencing must be accurate and provide all necessary details to allow your reader to locate the source. It is therefore a good idea to keep careful records of all the sources you accessed when researching your assignment. This way, you do not have to hunt for these details after you have finished writing.  

How to incorporate the ideas of others into your essay

It can be difficult for new academic writers to know how to incorporate others’ work into their own writing. By learning how to use quotations effectively, and how to summarise and paraphrase the words and ideas of others, you can better avoid unintentional plagiarism. 

A quotation is a word-for-word reproduction of someone else’s words, either spoken or written. When quoting from another source, you must: 

• Exception: For long quotations (e.g. over 40 words in APA or over 30 words in Harvard), indent the quotation instead of using quotation marks. The quotation should be introduced by a colon and followed by a citation. 
• Use quotation marks even if only borrowing a single phrase or word from another source. 
• Exception: If the source does not have numbered pages (e.g. a website, an interview), no page number is needed. However, if there is some other way of pointing to the specific location from which the quotation was taken (e.g. paragraph number, clause number, line in transcript), include that in the citation. 

Quotations should be logically integrated into your text. One way to do this is to lead into the quotation or paraphrase by using the author’s name (e.g. ‘According to Lines,’) followed by the quotation from Lines or a summary of Lines’s ideas. 

Quotations must fit grammatically into your text. It is allowable to modify quotations slightly to ensure a good fit. However, it is essential that these changes are clearly marked using square brackets ([ ]). It is also possible to omit words from a quotation, shown using an ellipsis (…). Note that if you omit words, you must be sure that the original meaning of the quotation is retained. You should never omit words to change the meaning of a quotation. 

The below examples show ways to integrate the original quotation ‘Most of the time, they don’t, and I mean really don’t, behave well’, showing changes to 1) the verb and 2) a pronoun. Notice the use of the square brackets to show your modifications to the quotation, and the ellipsis to show omitted words. 

• The teacher reported that the children were not ‘behav[ing] well’.
• According to the teacher, ‘Most of the time, [the children] don’t … behave well’. 

Finally, you should avoid using quotations that have not been adequately introduced. If a quotation is inserted without appropriate integration into your text, this can negatively affect the logical and grammatical flow of your work, and lower the quality of your writing. Not introducing quotations or incorporating them into your own sentences usually also means you are relying too heavily on the words of others, and your grades can suffer as a result.

Summarising and paraphrasing

Another option for integrating others’ ideas into your own assignments is by summarising and paraphrasing. Summarising means giving an overview of the main ideas in condensed form. Paraphrasing means putting an idea (usually in detail) into your own words.  

To summarise or paraphrase well, you need to read carefully and understand the ideas in the source. Then, you can think about what those ideas mean in the context of your assignment and write them in your own words, integrating them well into your own writing. If you take sentences completely from the original source and just change a few words, this is not paraphrasing, and may be considered plagiarism. 

For some students, the temptation to use a source’s original wording is high. To avoid this, after reading and understanding the author’s ideas, write just the keywords on a separate piece of paper. See if you can change some of the keywords to other words, while keeping the original meaning. Then, think about whether you can reorganise the order of the keywords, to write sentences that keep the original meaning, but that are quite different to the original. Using your keywords, and without referring to the original source, write your new sentences. It takes a while at first, but the process becomes automatic with practice. 

The importance of writing in your own words

Putting others’ work into your own words will not only ensure the material is effectively integrated into your writing, it also demonstrates to your reader (e.g. your lecturer) that you have understood, absorbed and interpreted the information. This is a key purpose of essay writing at university and will help you to get a better grade. In addition, the better you get at putting complex ideas into your own words, the more developed your writing style will become. 

Acknowledge every source

Remember that the need to reference is not limited to academic sources like books and journal articles. You need to reference ALL words, ideas or information taken from ANY source. 

These sources might include: 

• books and journal articles
• newspapers and magazines
• pamphlets or brochures
• films, documentaries, television programs or advertisements
• computer programs
• diagrams, illustrations, charts or pictures
• letters or emails
• personal interviews
• lecturers or tutors. (This is not always necessary, but check with your lecturer or tutor about his or her preferences before you draw on his or her ideas.)

Note that if the source you are citing is retrievable (i.e. can be located by another person using the information you provide in the reference list), you must provide a reference for the source. However, if the source is only available to you (e.g. a personal interview or email, or a private Facebook post), you should cite all necessary details in the text, but should not provide a reference in the reference list. ONLY irretrievable sources are not included in the reference list, and even these are still cited in the text. 

The only times you would not reference are:

• when referring to your own observations (e.g. a report on a field trip) or experiment results
• when writing about your own experiences (e.g. a reflective journal)
• when writing your own thoughts, comments or conclusions in an assignment
• when evaluating or offering your own analysis (e.g. parts of a critical review)
• when using ‘common knowledge’ (facts that can be found in numerous places and are likely to be known by a lot of people) or folklore
• when using generally accepted facts or information (this will vary in different disciplines of study. If in doubt, ask your tutor).

If you are concerned that you may not have referenced correctly, you should ask your tutor, lecturer or Academic Learning Advisor for their advice before submitting your assignment. Capstone Editing can also edit your work to correct your referencing and provide advice about how to reference correctly in the future.   

Other guides you may be interested in

Essay writing: everything you need to know and nothing you don&rsquo;t&mdash;part 1: how to begin.

This guide will explain everything you need to know about how to organise, research and write an argumentative essay.

Essay Writing Part 2: How to Organise Your Research

Organising your research effectively is a crucial and often overlooked step to successful essay writing.

Located in northeastern New South Wales 200 kilometres south of Brisbane, Lismore offers students a good study–play balance, in a gorgeous sub-tropical climate.

Rockhampton

The administrative hub for Central Queensland, Rockhampton is a popular tourist attraction due to its many national parks and proximity to Great Keppel Island.

How to Write an Academic Essay with References

Communicating research effectively through academic papers is one of the most essential components of any academic journey. While many PhD students and early career researchers are aware of presenting their data in a technical manner, either in the form of protocols or SOPs, writing an academic paper is a completely different process, since it needs to be backed with citations and references. Many of you may often be bogged with the question of what is an academic reference in an essay and how exactly it differs from a citation in academic writing. Since these two terms are often used interchangeably in the context of academic writing, there is a lot of confusion regarding their usage. This article provides some insights on how to write an academic essay and how to use citations and references in academic writing.

Both citations as well as references essentially serve the same purpose – that is, to point a reader toward the original source of a particular piece of information mentioned in your academic text. A citation in an academic writing is an acknowledgement of the source of information, and is present within the main text. On the other hand, references in academic writing are presented as a list of sources toward the end of the text. While understanding how to write an academic essay with references, it is important to know that each in-text citation is necessarily linked to a source in the list of references.

Now that we have had a brief insight into the basic similarities and differences between citations and references, here are some simple pointers on how to write an academic essay with the appropriate use of citations and references.

• Context and content-based usage of citations and references in academic writing

While it is essential to lead a reader toward the original source of the information, you need to maintain discretion and know whether that piece of information really warrants the usage of citations and/or references. For example, if you are stating facts it is not necessary to have it linked to a source in the list of references in academic writing. Conversely, even when you use a citation in academic writing, you need to refrain from copying the sentences directly from the original source in your text, as that will still be flagged as plagiarism. If you are combining information from two or more sources in the same paragraph, it is better to add all citations together at the end of that paragraph, rather than citing the source for each individual sentence – this will lend a non-cluttered look to your academic essay and help the reader gain all the necessary information with minimal distractions.

• Usage of citations and references based upon journal style and requirements

Usually journals have their own style and format for using a citation in academic writing. You can cite a source using numbers, wherein each number corresponds to a particular source in the list of references in your text. Or, you can also cite using the first/last author of the source mentioned in the list of references. Similarly, the amount of information that needs to be mentioned for each source in the list of references also varies depending upon the requirement and style of the journal. Additionally, some journals have an upper limit for the total number of references that can be used while describing information the text. Therefore, one important thing to keep in mind while compiling references in academic writing is to check the journal style and requirements thoroughly. This will also ensure that the final formatting process of your academic essay becomes comparatively smoother.

• Excessive number of citations and references in academic writing

If the information that you want to mention in your text is a part of two or more sources, it is usually not essential to add all the sources, as this will lead to cluttering and also add limitations to your usage of references, especially if your journal has an upper limit. You can choose one or two sources that could be most efficient in leading the reader toward the full information based upon how effectively the sources cover the information. You can also make the choice based upon the impact factor of the journal in which the original source is published. Citing information from a higher impact factor journal, provided it conveys the same information, could also help in elevating the quality of your academic paper to a certain extent.

We hope that the above pointers help you in improving your skills at academic writing and provide you with some clarity on how to write an academic essay with references and citations.

Related Reads:

• 6 Simple Steps to Convert a PhD Thesis Into a Journal Article
• How to Write a Research Paper Title
• Good Writing Habits: 7 Ways to Improve Your Academic Writing

5 Reasons for Rejection After Peer Review

How to avoid desk rejection: stop making these 5 mistakes, what are synonyms and how do you use synonyms in academic writing , you may also like, ai in education: it’s time to change the..., publish research papers: 9 steps for successful publications , self-plagiarism in research: what it is and how..., 6 tips for post-doc researchers to take their..., 8 most effective ways to increase motivation for..., how to make your thesis supervision work for..., how to write a conclusion for research papers..., ethical research practices for research with human subjects, what is peer review: importance and types of....

Scribbr Referencing Generator

Accurate Harvard, APA, MLA, and Chicago references, verified by experts, trusted by millions.

Reference sources in seconds with Autocite

Look up your source by its title, URL, ISBN, or DOI, and let Scribbr do the rest! The reference generator will automatically find all the necessary information to generate a perfect reference, including the author(s), publication date, and publisher.

Perfectly formatted references every time

Inaccurate references can cost you points on your assignments, so our seasoned referencing experts have invested countless hours in perfecting Scribbr’s reference generator algorithms. We’re proud to be recommended by teachers and universities across the UK.

Enjoy a referencing generator without flashy ads

Staying focused is already difficult enough, so unlike other reference generators, Scribbr won’t slow you down with flashing banner ads and video pop-ups. That’s a promise!

Citation Generator features you'll love

Look up your source by its title, URL, ISBN, or DOI, and let Scribbr find and fill in all the relevant information automatically.

Harvard, APA, MLA, Chicago

Generate flawless references according to the official Harvard , APA , MLA, or Chicago style rules. More referencing styles will be available soon!

Export to Word

When your reference list is complete, export it to Word. We’ll apply the official formatting guidelines automatically.

Lists and folders

Create separate reference lists for each of your assignments to stay organized. You can also group related lists into folders.

Export to Bib(La)TeX

Are you using a LaTex editor like Overleaf? If so, you can easily export your references in Bib(La)TeX format with a single click.

Custom fonts

Change the typeface used for your reference list to match the rest of your document. Options include Times New Roman, Arial, and Calibri.

Industry-standard technology

Scribbr’s Referencing Generator is built using the same citation software (CSL) as Mendeley and Zotero, but with an added layer for improved accuracy.

Annotations

Describe or evaluate your sources in annotations, and Scribbr will generate a perfectly formatted annotated bibliography.

Referencing guides

Scribbr’s popular guides and videos will help you understand everything related to finding, evaluating, and referencing sources.

Secure backup

Your work is saved automatically after every change and stored securely in your Scribbr account.

• Introduction
• Finding sources

Evaluating sources

• Integrating sources

Referencing sources

Tools and resources, a quick guide to working with sources.

Working with sources is an important skill that you’ll need throughout your academic career.

It includes knowing how to find relevant sources, assessing their authority and credibility, and understanding how to integrate sources into your work with proper referencing.

This quick guide will help you get started!

Finding relevant sources

Sources commonly used in academic writing include academic journals, scholarly books, websites, newspapers, and encyclopedias. There are three main places to look for such sources:

• Research databases: Databases can be general or subject-specific. To get started, check out this list of databases by academic discipline . Another good starting point is Google Scholar .
• Your institution’s library: Use your library’s database to narrow down your search using keywords to find relevant articles, books, and newspapers matching your topic.
• Other online resources: Consult popular online sources like websites, blogs, or Wikipedia to find background information. Be sure to carefully evaluate the credibility of those online sources.

When using academic databases or search engines, you can use Boolean operators to refine your results.

Generate Harvard, APA, MLA, and Chicago style references in seconds

Get started

In academic writing, your sources should be credible, up to date, and relevant to your research topic. Useful approaches to evaluating sources include the CRAAP test and lateral reading.

CRAAP is an abbreviation that reminds you of a set of questions to ask yourself when evaluating information.

• Currency: Does the source reflect recent research?
• Relevance: Is the source related to your research topic?
• Authority: Is it a respected publication? Is the author an expert in their field?
• Accuracy: Does the source support its arguments and conclusions with evidence?
• Purpose: What is the author’s intention?

Lateral reading

Lateral reading means comparing your source to other sources. This allows you to:

• Verify evidence
• Contextualize information
• Find potential weaknesses

If a source is using methods or drawing conclusions that are incompatible with other research in its field, it may not be reliable.

Integrating sources into your work

Once you have found information that you want to include in your paper, signal phrases can help you to introduce it. Here are a few examples:

Following the signal phrase, you can choose to quote, paraphrase or summarize the source.

• Quoting : This means including the exact words of another source in your paper. The quoted text must be enclosed in quotation marks or (for longer quotes) presented as a block quote . Quote a source when the meaning is difficult to convey in different words or when you want to analyze the language itself.
• Paraphrasing: This means putting another person’s ideas into your own words. It allows you to integrate sources more smoothly into your text, maintaining a consistent voice. It also shows that you have understood the meaning of the source.
• Summarizing : This means giving an overview of the essential points of a source. Summaries should be much shorter than the original text. You should describe the key points in your own words and not quote from the original text.

Whenever you quote, paraphrase, or summarize a source, you must include a citation crediting the original author.

Referencing your sources is important because it:

• Allows you to avoid plagiarism
• Establishes the credentials of your sources
• Backs up your arguments with evidence
• Allows your reader to verify the legitimacy of your conclusions

The most common citation styles in the UK are APA, MLA, Harvard, Vancouver, MHRA, and Oscola. Each citation style has specific rules for formatting citations.

Scribbr’s free Reference Generator can generate perfect references and in-text citations in both APA and MLA styles. More citation styles will be available soon!

Scribbr and partners offer tons of tools and resources to make working with sources easier and faster. Take a look at our top picks:

• Reference Generator: Automatically generate Harvard and APA references .
• Plagiarism Checker : Detect plagiarism in your paper using the most accurate Turnitin-powered plagiarism software available to students.
• Proofreading services : Have a human editor improve your writing.
• Knowledge Base : Explore hundreds of articles, bite-sized videos, time-saving templates, and handy checklists that guide you through the process of research, writing, and citation.

• Skip to main content
• All Articles
• Study abroad info Articles
• Study abroad info Applying to university

• Choosing where to study
• Choosing a subject
• Financing your studies
• Getting your visa
• Before you leave
• Housing & accommodation
• Once you arrive
• Post-study life

How to reference in an academic essay

Writing a good academic essay which includes quotes and references is a vital writing skill to have at university. We give you useful vocabulary to help you understand what referencing is, why it’s important and how to use it.

When you start studying abroad at university you will have to write several essays for each of your modules. An important part of essay writing is referencing. It's critical to learn what it is, why you must use it and how to use it. Read on to find the answers to these questions. Before going into more detail, let’s first look at some useful vocabulary to help you a little later on.

Useful vocabulary 

bibliography  – This is like a reference list as it includes a list of all the sources  (see below)  you have cited  (see below)  in your essay, plus any sources you used in your research on the topic. You may not have cited the sources in your essay, but you used them in your research. 

citation (noun)  – This is a quotation from, or a reference to, a book, journals, and so on.  (verb: to cite)

in-text   citation/referencing  – This is a reference to a book or journal in the actual text of the essay. This is different from a reference list or bibliography.

date of publication  – This is the date the source you have used was published (made available either in printed format or online).

reference  -  (verb and noun)  This is where you acknowledge or recognise the importance of the sources you have used in the writing of your essay. There are many ways to reference the sources used (more on this later). 

reference list  – This is a detailed list of the sources  (see definition below)  you have cited in your essay. The reference list is added at the end of your essay and is listed in alphabetical order. 

source  – (noun) These are the materials you have used to gather information, data, facts, and statistics. Sources must be credible (they can be trusted and have authority). They can be journals, academic books, conference papers, and from credible and trustworthy websites such as .gov / .edu / .ac.

Why do I have to use references in my essay?

It is important to acknowledge what sources you have used in the writing of your essay to show that you have done a great deal of reading on the subject and provide you with evidence to support your point of view. 

Equally importantly, referencing makes clear which ideas and words are your own and which ideas and/or words have been taken from someone else. When you use other people’s words without acknowledging the source this is called plagiarism. 

Plagiarism is a form of copying that has very strong negative consequences such as failing your course, lower grades and even being expelled from university. Each university and each faculty have their own rules of what constitutes plagiarism, and there are software tools that professors can use to see if a student has copied the work and words of another person. 

The best way to avoid plagiarism while note-taking and writing is to improve your  essay writing skills  and learn how to paraphrase and reference correctly. 

How do I write a reference correctly?

This question is a little more difficult to answer as different referencing systems are better suited to certain subjects than to others. Each course will state what referencing system they require students to use. 

Here are some referencing systems that are used by universities in the  US ,  Canada ,  Australia ,  New Zealand ,  Ireland  and the  UK : 

• APA  – American Psychological Association – this style is commonly used in psychology, health and social sciences and medicine. 
• Chicago  – this style is mainly used in humanities subjects. 
• Harvard  – this style has been adapted and used by many universities, faculties, and course programmes. It’s important to check your course programme and follow it carefully. 

There are many more, but we’ve listed some of the most common systems. Here is what these styles look like in-text and in a reference list/bibliography:

Your university will have detailed style guides for you to follow.  International students also have a great deal of help on offer at university from the language support services and study skills tutorials and resources online.

The good news is that learning how to use referencing and citations correctly isn’t a challenge just for international students. This is experienced by all new university students, so don’t be embarrassed to ask for help.

Now that you’re informed about a critical skill needed when studying at university, you can continue your research. You can discover the top 10 reasons to study abroad ,  get in touch with a counsellor for more guidance, or find a course that may suit you.

IELTS vs. TOEFL: Which should you take?

All international students who want to study in an English-speaking country must show they have the required level of English. There are a few English language exams that are accepted by universities all over the world. We’re going to focus on two of these: IELTS and TOEFL.   Before we continue, let’s look at what these names mean. IELTS is the International English Language Testing System. TOEFL is the Test of English as a Foreign Language.

Top 20 FAQs about the IELTS exam

What is the IELTS exam? The International English Language Testing System (IELTS) is an English language exam which is recognised worldwide by more than 11,000 universities, employers and immigration government bodies. The IELTS test explained   What are the benefits of the IELTS test? As mentioned above, the IELTS exam is recognised all over the world and is used for university education and immigration purposes. It’s

Applying to university: Essential documents you need

You’ll learn as an international student, that nothing can be done unless you can successfully prove who you are. In countries where immigration policy and security are of utmost importance, this can be easier said than done, with long procedures involving lots of paperwork and waiting.   Originals vs. Copies However, you can make things a lot easier for yourself if you keep to hand a file containing the following important documentation. This

What are the benefits of the IELTS test?

As a prospective international student, you’ll know the importance of demonstrating your English language proficiency as part of the university application process. You’ll also know that several English language tests can be included as part of the entry requirements. With all of these options available, you may wonder what sets them apart, and what makes IELTS so popular. We're here to answer these questions and more, explaining the benefits of an IELTS test.

Get in touch

• Free Tools for Students
• Harvard Referencing Generator

Free Harvard Referencing Generator

Generate accurate Harvard reference lists quickly and for FREE, with MyBib!

🤔 What is a Harvard Referencing Generator?

A Harvard Referencing Generator is a tool that automatically generates formatted academic references in the Harvard style.

It takes in relevant details about a source -- usually critical information like author names, article titles, publish dates, and URLs -- and adds the correct punctuation and formatting required by the Harvard referencing style.

The generated references can be copied into a reference list or bibliography, and then collectively appended to the end of an academic assignment. This is the standard way to give credit to sources used in the main body of an assignment.

👩‍🎓 Who uses a Harvard Referencing Generator?

Harvard is the main referencing style at colleges and universities in the United Kingdom and Australia. It is also very popular in other English-speaking countries such as South Africa, Hong Kong, and New Zealand. University-level students in these countries are most likely to use a Harvard generator to aid them with their undergraduate assignments (and often post-graduate too).

🙌 Why should I use a Harvard Referencing Generator?

A Harvard Referencing Generator solves two problems:

• It provides a way to organise and keep track of the sources referenced in the content of an academic paper.
• It ensures that references are formatted correctly -- inline with the Harvard referencing style -- and it does so considerably faster than writing them out manually.

A well-formatted and broad bibliography can account for up to 20% of the total grade for an undergraduate-level project, and using a generator tool can contribute significantly towards earning them.

⚙️ How do I use MyBib's Harvard Referencing Generator?

Here's how to use our reference generator:

• If citing a book, website, journal, or video: enter the URL or title into the search bar at the top of the page and press the search button.
• Choose the most relevant results from the list of search results.
• Our generator will automatically locate the source details and format them in the correct Harvard format. You can make further changes if required.
• Then either copy the formatted reference directly into your reference list by clicking the 'copy' button, or save it to your MyBib account for later.

MyBib supports the following for Harvard style:

🍏 What other versions of Harvard referencing exist?

There isn't "one true way" to do Harvard referencing, and many universities have their own slightly different guidelines for the style. Our generator can adapt to handle the following list of different Harvard styles:

• Cite Them Right
• Manchester Metropolitan University (MMU)
• University of the West of England (UWE)

Daniel is a qualified librarian, former teacher, and citation expert. He has been contributing to MyBib since 2018.

• Email & Portal

Tools for Effective Writing

• Academic Writing
• Step 1: Before You Start Writing
• Step 2: Writing the Literature Review
• Examples and Additional Resources
• Writing an Annotated Bibliography
• References vs Bibliography
• Citation Guide
• Writing for Nursing
• Grammarly This link opens in a new window
• Interlibrary Loan This link opens in a new window

Citing Sources - References vs Bibliography

References and Bibliographies - What's the difference?

When you write academic papers, you will need to include a list of sources you used to write the paper. There are two main ways to list your sources, with a reference list or a bibliography.

References include sources that have been directly cited in your paper. For each source, you will have at least one in-text citation in the body of your paper. The citation styles that use reference lists include APA citations, AMA citations, and MLA citations.

Bibliographies, on the other hand, contain all the sources that you have used for your paper, whether they are directly cited or not. In a bibliography, you should include all of the materials you consulted in preparing your paper. Chicago citations and Oxford citations are two citation styles that use bibliographies.

Both reference lists and bibliographies appear at the end of a written work and are usually organized alphabetically. A paper can have both a reference list and a bibliography.

For more information on how to cite your sources, check out the De Paul Library's Citation Guide .

• << Previous: Examples and Additional Resources
• Next: Citation Guide >>
• Last Updated: Jul 26, 2023 2:17 PM
• URL: https://stmary.libguides.com/toolsforeffectivewriting

This paper is in the following e-collection/theme issue:

Published on 5.4.2024 in Vol 26 (2024)

Evaluation of Large Language Model Performance and Reliability for Citations and References in Scholarly Writing: Cross-Disciplinary Study

Authors of this article:

Original Paper

• Joseph Mugaanyi 1 * , MBBS, MD   ; 
• Liuying Cai 2 * , MPhil   ; 
• Sumei Cheng 2 , PhD   ; 
• Caide Lu 1 , MD, PhD   ; 
• Jing Huang 1 , MD, PhD  

1 Department of Hepato-Pancreato-Biliary Surgery, Ningbo Medical Center Lihuili Hospital, Health Science Center, Ningbo University, Ningbo, China

2 Institute of Philosophy, Shanghai Academy of Social Sciences, Shanghai, China

*these authors contributed equally

Corresponding Author:

Jing Huang, MD, PhD

Department of Hepato-Pancreato-Biliary Surgery, Ningbo Medical Center Lihuili Hospital

Health Science Center

Ningbo University

No 1111 Jiangnan Road

Ningbo, 315000

Phone: 86 13819803591

Email: [email protected]

Background: Large language models (LLMs) have gained prominence since the release of ChatGPT in late 2022.

Objective: The aim of this study was to assess the accuracy of citations and references generated by ChatGPT (GPT-3.5) in two distinct academic domains: the natural sciences and humanities.

Methods: Two researchers independently prompted ChatGPT to write an introduction section for a manuscript and include citations; they then evaluated the accuracy of the citations and Digital Object Identifiers (DOIs). Results were compared between the two disciplines.

Results: Ten topics were included, including 5 in the natural sciences and 5 in the humanities. A total of 102 citations were generated, with 55 in the natural sciences and 47 in the humanities. Among these, 40 citations (72.7%) in the natural sciences and 36 citations (76.6%) in the humanities were confirmed to exist ( P =.42). There were significant disparities found in DOI presence in the natural sciences (39/55, 70.9%) and the humanities (18/47, 38.3%), along with significant differences in accuracy between the two disciplines (18/55, 32.7% vs 4/47, 8.5%). DOI hallucination was more prevalent in the humanities (42/55, 89.4%). The Levenshtein distance was significantly higher in the humanities than in the natural sciences, reflecting the lower DOI accuracy.

Conclusions: ChatGPT’s performance in generating citations and references varies across disciplines. Differences in DOI standards and disciplinary nuances contribute to performance variations. Researchers should consider the strengths and limitations of artificial intelligence writing tools with respect to citation accuracy. The use of domain-specific models may enhance accuracy.

Introduction

In the ever-evolving landscape of scholarly research and academic discourse, the role of technology in aiding and enhancing the research process has grown exponentially. One of the most notable advancements in this regard is the emergence of large language models (LLMs) such as GPT-3.5, which have demonstrated impressive capabilities in generating written content across various domains, including academic writing. These LLMs, powered by vast corpora of text data and sophisticated machine-learning algorithms, have offered researchers and writers a new tool for assistance in crafting scholarly documents [ 1 - 3 ]. LLMs were initially designed and developed to primarily assist in natural language writing. However, since the release of ChatGPT in late 2022, the tool has been adopted in a wide range of scenarios, including customer care, expert systems, as well as literature searches and academic writing. Researchers have already used LLMs to write their academic papers, as demonstrated by Kishony and Ifargan [ 4 ]. While the potential of these tools is evident, it is essential to critically assess their performance, especially in the intricate domains of citations and references, which are the foundation of academic discourse and credibility.

Citations and references serve as the backbone of scholarly communication, providing the necessary context, evidence, and credit to prior works, thus fostering intellectual dialogue and ensuring the integrity of the research process. Accuracy in generating citations and the inclusion of Digital Object Identifiers (DOIs) [ 5 ] are paramount, as they directly influence the traceability and accessibility of cited works. Despite the promise of LLMs, concerns have emerged regarding the reliability and precision of their generated citations and references, raising questions about their suitability as academic writing assistants. Studies on the viability of LLMs as writing assistants in scholarly writing [ 6 - 8 ] underscore the significance of this body of research within the broader academic landscape. Although prior works are quite informative [ 9 - 12 ], there is a lack of an interdisciplinary perspective on citations and references generated by LLMs, which is vital for understanding how LLMs perform across different disciplines.

An increasing number of academics and researchers, especially in countries where English is not a first language (eg, China), are relying on ChatGPT to translate their work into English, research the existing published literature, and even generate citations and references to published literature. Therefore, the aim of this study was to evaluate LLM performance in generating citations and references across two distinct domains, the natural sciences and humanities, by assessing both the presence and accuracy of citations, the existence and accuracy of DOIs, and the potential for hallucination. We aim to provide valuable insights into the strengths and limitations of LLMs in supporting academic writing in diverse research contexts.

The outcomes of this study will contribute to a nuanced understanding of the capabilities and limitations of LLMs as academic writing assistants. Moreover, our findings may inform best practices for researchers and writers who employ these tools in their work, fostering transparency and accuracy in scholarly communication.

LLM Concepts

An LLM is a catch-all term for a machine-learning model designed and trained to understand and generate natural language. LLMs are considered “large” language models due to the sheer number of parameters in the model. A parameter in machine learning is a numerical variable or weight that is optimized through training to map a relationship between the input and the output. LLMs have millions to billions of parameters.

Current LLMs are mostly based on the transformer architecture ( Figure 1 ). However, before transformers were introduced in 2017 [ 13 ], recurrent neural nets (RNNs) were mostly used for natural language processing. One key limitation of RNNs was the length of text they could handle. In 2015, Bahdanau et al [ 14 ] proposed accounting for attention to improve RNN performance with long text. Drawing inspiration for the RNN’s encoder-decoder design, the transformer consists of an encoder and a decoder; however, unlike the RNN, the transformer does not perform sequential data processing and each layer can address all other layers. This allows the transformer model to handle different parts of the input as it processes each part at different stages. This is the mechanism that allows for self-attention in the transformer model.

The way attention works in a transformer model is by computing attention weights for each token, and then the relevance of the token is determined based on the weights. This allows the model to track and assign hierarchical values to each token. Fundamentally, this is similar to how humans process language by extracting the key details out of a chunk of text. This architecture is the linchpin for the majority of LLMs, including the GPT model [ 15 ] that is the basis of OpenAI’s ChatGPT or the bidirectional encoder representations from transformers (BERT) algorithm [ 16 ]. These are broadly categorized into encoder-style and decoder-style transformers, with the former mostly applying to predictive tasks and the latter applying to generative tasks.

Irrespective of the architecture, as an encoder-style or decoder-style transformer, the model is trained on a vast volume of data. The objective is to train a model capable of applying the knowledge gained from the training data to unseen data or situations. This is referred to as generalization. If the model is capable of precise recall of data it has previously been exposed to, this would be memorization and overfitting is said to have occurred. However, this does not mean that memorization is in itself a negative feature. Indeed, there are situations where memorization is preferable to generation such as in the task of information cataloging.

LLMs in Academia

LLMs can handle tasks such as text classification, translation, summarization, and text generation. Since the advent of the internet, and with it the publication of scientific information online, the amount of global academic output exploded, with more than 5 million articles published in 2022 ( Table 1 ). Given the pressure in academia to keep up with developments in one’s field, it is increasingly becoming more difficult to track, prioritize, and keep up with scientific information. It is against this backdrop that LLMs offer an opportunity. Perhaps the most obvious use case is in literature reviews and summarization, reference lookup, and data generation.

However, there are still several questions that need to be answered. First, machine-learning models are inherently probabilistic, meaning that they are not deterministic. Therefore, for the same user input, the model may give different results due to the variability baked into the model. While this can be a valuable trait for creative endeavors, in academic and scientific works, there is a need for reproducibility and reliability, and it remains unclear how well this can be achieved. Second, LLMs are constrained to the information they are trained on. This can be affected by selection bias, the quality of data used, artifacts resulting from data cleaning, and other factors. In essence, we rely on trusting the trainer to provide accurate and unbiased training data to the models.

There is potential for LLMs to be useful tools for delivering academic and scientific information to various audiences, including—but not limited to—students and other academics. However, for this use case, a degree of memorization of the underlying content is necessary. Where information is unviable, it would be better to state so rather than to interpolate. In the current iteration of LLMs, since the training is geared toward generalization and the models are probabilistic, they tend to interpolate and fill in the missing information with synthetic text. There is still a need to explore this process deeper to find solutions.

Data Collection and Validation

Topics were selected and categorized as either natural sciences or humanities. Topics were included if they were: (1) clinical or biomedical–related research in the natural sciences category and philosophy/psychology-related research in the humanities category, and (2) published in English. Topics were excluded if they were: (1) not in English, (2) related to a highly specialized or niche field, and (3) sensitive or controversial in nature. Two researchers independently prompted ChatGPT (GPT-3.5) to write sections of a manuscript while adhering to the American Psychological Association style [ 17 ] for citations and including the DOI of each reference. Citations and references generated by ChatGPT were collected for subsequent analysis. The researchers then independently validated the references by conducting searches on Google Scholar, PubMed, and Google Search for each cited reference. The primary objective was to confirm the existence and accuracy of the cited literature. DOI existence and validation were confirmed using the DOI Foundation website [ 18 ]. DOIs that did not exist or were matched to a different source were considered hallucinations [ 19 ]. Data collected by both researchers were aggregated and compared. Independent validation was performed to ensure agreement between the two researchers regarding the existence, validity, and accuracy of the citations and DOIs. Any disagreements or discrepancies were resolved through discussion and consensus.

In this study, hallucination refers to instances where ChatGPT 3.5 generates DOIs and/or citations that do not correspond to actual, valid DOIs/citations for scholarly references. In these instances, the model may produce DOIs and/or citations that seem authentic but are in fact incorrect or nonexistent. The Levenshtein distance, also known as the edit distance, is a measure of the similarity between two strings by calculating the minimum number of single-character edits (insertions, deletions, or substitutions) required to transform one string into the other. In other words, this metric quantifies the “distance” between two strings in terms of the minimum number of operations needed to make them identical. We used the Levenshtein distance to compare the DOI generated by ChatGPT with the correct DOI. This comparison helps to measure how closely the artificial intelligence (AI)–generated DOI aligns with the expected DOI for a given citation. By calculating the Levenshtein distance, we can quantify the differences between the AI-generated DOI and the correct DOI. Larger Levenshtein distance values suggest greater dissimilarity, indicating potential inaccuracies in the AI-generated DOI.

Statistical Analysis

Data analysis was conducted using SPSS 26 and Python. The Levenshtein distance [ 20 ] between the generated DOI and the actual DOI was calculated using the thefuzz package in Python to quantitatively assess the DOI accuracy. Continuous variables are reported as mean (SD) and categorical variables are presented as absolute numbers and percentages. An independent-sample t test was used to compare continuous variables, whereas the Fisher exact test was used for comparisons of categorical variables. A P value <.05 was considered statistically significant in all tests.

Ethical Considerations

This study was exempt from ethical review since no animal or human participants were involved.

Included Topics and Citations

Ten manuscript topics were selected and included in the study, with 5 in the natural sciences group and 5 in the humanities group. ChatGPT 3.5 was prompted to write an introduction section for each topic between July 10 and August 15, 2023. A total of 102 citations were generated by ChatGPT. Of these, 55 were in the natural sciences group and 47 in the humanities group. The existence, validity, and relevance of citations were examined irrespective of the corresponding DOIs. The results are summarized in Table 2 . A list of the included topics and a sample of prompts to ChatGPT are provided in Multimedia Appendix 1 .

a Categorical variables were compared using the Fisher exact test; the continuous variable (Levenshtein distance) was compared using the independent-sample t test.

b DOI: Digital Object Identifier.

Citation Existence and Accuracy

Of the 102 generated citations, 76 (74.5%) were found to be real and exist in the published literature, with 72.7% and 76.6% of the citations verified in the natural and humanities group, respectively. There was no significant difference between the two groups ( P =.42), indicating that the validity of the citations was relatively consistent between the two domains. Similarly, when assessing the accuracy of the citations, no significant difference was observed ( Table 2 ).

Citation Relevance

The relevance of citations generated by ChatGPT was evaluated by assessing whether they were appropriate and contextually meaningful within the research topics. Our analysis indicated that 70.9% and 74.5% of citations in the natural sciences and humanities categories were deemed relevant, respectively ( Table 2 ). The difference was not statistically significant ( P =.43), suggesting that ChatGPT demonstrated a similar ability to generate contextually relevant citations in both domains.

DOI Existence, Accuracy, and Hallucination

Our analysis revealed significant differences between the two domains with respect to DOIs. In the natural sciences, 70.9% of the included DOIs were real, whereas in the humanities, only 38.3% of the DOIs generated were real ( P =.001; Table 2 ). Similarly, the level of DOI accuracy was significantly higher for the natural sciences than for the humanities ( P =.003). Moreover, the occurrence of DOI hallucination, where ChatGPT generates DOIs that do not correspond with the existing literature, was more prevalent in the humanities than in the natural sciences ( P =.001). The mean Levenshtein distance, which measures the deviation between the generated DOI and the actual DOI, was significantly higher in the natural sciences group than in the humanities ( P =.009; Table 2 ).

Principal Findings

The results of this study shed light on the performance of ChatGPT (GPT-3.5) as an academic writing assistant in generating citations and references in natural sciences and humanities topics. Our findings reveal notable differences in the accuracy and reliability of the citations and references generated by ChatGPT when applied to natural sciences and humanities topics. Hallucination in the context of LLMs such as ChatGPT refers to a phenomenon where the model generates content that is incorrect, fabricated, or not grounded in reality. Hallucination occurs when the model produces information that appears plausible or contextually relevant but lacks accuracy or fidelity to real-world knowledge.

The most striking observation was the significant disparity in the existence and accuracy of the DOIs between the two domains. In natural sciences topics, DOIs were real in 70.9% of the generated citations, representing a significantly higher rate compared to the low rate of 38.3% real DOIs in the humanities topics. The discrepancies in the DOI existence and accuracy in the two domains may be attributed to the differential adoption and availability of DOIs across academic disciplines, where the natural sciences literature has often been more proactive in adopting the DOI system of referencing and linking to scholarly works than the humanities. It is a general practice that journals publishing on the natural sciences frequently mandate DOI inclusion, whereas publishers in the humanities have been slower to adopt such standards [ 21 , 22 ]. Consequently, the performance of the ChatGPT LLM in generating accurate DOIs appears to reflect these disciplinary disparities.

LLMs may generate fictional “facts” presented as true “real-world facts,” which is referred to as hallucination [ 19 , 23 ]. In this study, we considered hallucination to have occurred if the DOI of the generated citation was not real or was real but was linked to a different source. DOI hallucination was more frequent in the humanities (89.4%) than in the natural sciences (61.8%). This finding may be explained by the broader and less structured nature of the humanities literature. There is also a high tendency to provide citations from books and other media that do not use DOIs in the humanities. Therefore, researchers in the humanities should not consider DOIs generated by ChatGPT. Even when ChatGPT generates DOIs for humanities citations, they are more likely to deviate from the correct DOI, potentially leading to the inability to access the cited sources and use the DOIs in citation management tools such as EndNote.

In contrast to the disparities observed in DOI-related metrics, our study found a remarkable consistency in the existence, validity, and relevance of the generated citations in the natural sciences and humanities, with real citations found 72.7% and 76.6% of the time and accurate citations confirmed in 67.3% and 61.7% of cases, respectively. This suggests that the citations generated by ChatGPT can be expected to be reliable approximately 60% of the time.

The divergent performance of ChatGPT between the natural sciences and humanities underscores the importance of considering disciplinary nuances when implementing AI-driven writing assistants in academic contexts. Researchers and writers in both domains should be aware of the strengths and limitations of such tools, particularly in relation to citation practices and DOI accuracy. Future research could delve deeper into the factors influencing DOI accuracy and explore strategies for improving DOI generation by LLMs in the humanities literature. Additionally, the development of domain-specific AI writing models may offer tailored solutions to enhance citation and reference accuracy in various academic disciplines.

In this study, we focused only on the potential use of LLMs in citations and references in scholarly writing; however, the scope to which these models are going to be adopted in academic works is much broader. We believe that these models will be improved over time and that they are here to stay. As such, our argument in this paper is not that LLMs should not be used in scholarly writing, but rather that in their iteration, we ought to be aware of their limitations, primarily concerning the reliability of not only the text they generate but also how they interpret that text.

Although the transformer models that are the foundation of LLMs are very capable of handling a significant amount of information, they still do have context-window limitations. The context window is the textual range or span of the input that the LLM can evaluate to generate a response at any given moment. As an example, GPT-3 has a context window of 2000 tokens, whereas GPT-4’s context window is 32,000 tokens. As such, since the size of the context window impacts model performance (larger is better), GPT-4 outperforms GPT-3 (at the cost of more computation and memory). In scientific knowledge, context is key. Removing a word from the context may greatly affect the information being conveyed. Therefore, we believe that the future of LLMs in academia will rely on fine-tuning the LLMs to capitalize on memorization where necessary, reproducibility and stability of the models, as well as access to the latest information rather than only the training data.

Limitations

There were several limitations to this study. The study included a limited number of topics (10 in total), which can only offer insight but cannot possibly cover the full spectrum of complexity and diversity within the two disciplines. Only ChatGPT 3.5 was prompted since it is the most widely used LLM for this purpose and has a free tier that the majority of users rely on. Newer models, including GPT-4, Claude+, and Google’s Gemini, may give significantly different results. Our study focused on the accuracy of citations and DOIs without an exploration of potential user feedback or subjective assessment of the overall quality and coherence of the generated content. These limitations can be addressed in future research.

In conclusion, our study provides valuable insights into the performance of ChatGPT in generating citations and references across interdisciplinary domains. These findings contribute to the ongoing discourse on the use of LLMs in scholarly writing, emphasizing the need for nuanced consideration of discipline-specific challenges and the importance of robust validation processes to ensure the accuracy and reliability of generated content.

Acknowledgments

This work was supported by the Municipal Key Technical Research and Development Program of Ningbo (2023Z160).

Data Availability

The data sets generated during and/or analyzed during this study are available from the corresponding author upon reasonable request.

Conflicts of Interest

None declared.

List of included topics and ChatGPT 3.5 prompt structure.

• Golan R, Reddy R, Muthigi A, Ramasamy R. Artificial intelligence in academic writing: a paradigm-shifting technological advance. Nat Rev Urol. Jun 2023;20(6):327-328. [ CrossRef ] [ Medline ]
• Checco A, Bracciale L, Loreti P, Pinfield S, Bianchi G. AI-assisted peer review. Humanit Soc Sci Commun. Jan 25, 2021;8(1):25. [ CrossRef ]
• Hutson M. Could AI help you to write your next paper? Nature. Nov 31, 2022;611(7934):192-193. [ CrossRef ] [ Medline ]
• Conroy G. Scientists used ChatGPT to generate an entire paper from scratch - but is it any good? Nature. Jul 2023;619(7970):443-444. [ CrossRef ] [ Medline ]
• Paskin N. Digital object identifier (DOI®) system. In: Encyclopedia of Library and Information Sciences. Milton Park. Taylor and Francis; 2010;1586-1592.
• Hosseini M, Horbach SPJM. Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review. Res Integr Peer Rev. May 18, 2023;8(1):4. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Santra PP, Majhi D. Scholarly communication and machine-generated text: is it finally AI vs AI in plagiarism detection? J Inf Knowl. Jul 01, 2023;60(3):175-183. [ CrossRef ]
• Esplugas M. The use of artificial intelligence (AI) to enhance academic communication, education and research: a balanced approach. J Hand Surg Eur Vol. Sep 07, 2023;48(8):819-822. [ CrossRef ] [ Medline ]
• Kumar AHS. Analysis of ChatGPT tool to assess the potential of its utility for academic writing in biomedical domain. BEMS Reports. Jan 05, 2023;9(1):24-30. [ CrossRef ]
• Marchandot B, Matsushita K, Carmona A, Trimaille A, Morel O. ChatGPT: the next frontier in academic writing for cardiologists or a pandora's box of ethical dilemmas. Eur Heart J Open. Mar 2023;3(2):oead007. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zheng H, Zhan H. ChatGPT in scientific writing: a cautionary tale. Am J Med. Aug 2023;136(8):725-726. [ CrossRef ] [ Medline ]
• Alkaissi H, McFarlane S. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. Feb 2023;15(2):e35179. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A, et al. Attention is all you need. arXiv. [ FREE Full text ]
• Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv. [ FREE Full text ]
• Radford A, Narasimhan K, Salimans T, Sutskever I. Improving language understanding by generative pre-training. OpenAI. 2018. URL: https://openai.com/research/language-unsupervised [accessed 2024-04-01]
• Devlin J, Chang M, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv. Preprint posted online on October 11, 2018. [ CrossRef ]
• Degelman D, Harris M. APA style essentials. Vanguard University. 2000. URL: https://www.vanguard.edu/uploaded/research/apa_style_guide/apastyleessentials.pdf [accessed 2024-04-01]
• DOI Foundation. URL: https://doi.org/ [accessed 2024-04-01]
• Salvagno M, Taccone FS, Gerli AG. Artificial intelligence hallucinations. Crit Care. May 10, 2023;27(1):180. [ CrossRef ] [ Medline ]
• Yujian L, Bo L. A normalized Levenshtein distance metric. IEEE Trans Pattern Anal Machine Intell. Jun 2007;29(6):1091-1095. [ CrossRef ]
• Eve MP. Open Access and the Humanities. Cambridge, UK. Cambridge University Press; 2014.
• Narayan B, Luca E, Tiffen B, England A, Booth M, Boateng H. Scholarly communication practices in humanities and social sciences: a study of researchers' attitudes and awareness of open access. Open Inf Sci. 2018;2(1):168-180. [ CrossRef ]
• Zhang Y, Li Y, Cui L, Cai D, Liu L, Fu T, et al. Siren's song in the AI ocean: a survey on hallucination in large language models. arXiv. [ FREE Full text ]

Abbreviations

Edited by A Mavragani; submitted 19.09.23; peer-reviewed by Y Bu, W Li, I Liu, A Mihalache; comments to author 08.12.23; revised version received 14.12.23; accepted 12.03.24; published 05.04.24.

©Joseph Mugaanyi, Liuying Cai, Sumei Cheng, Caide Lu, Jing Huang. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 05.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 26 March 2024

Predicting and improving complex beer flavor through machine learning

• Michiel Schreurs   ORCID: orcid.org/0000-0002-9449-5619 1 , 2 , 3   na1 ,
• Supinya Piampongsant 1 , 2 , 3   na1 ,
• Miguel Roncoroni   ORCID: orcid.org/0000-0001-7461-1427 1 , 2 , 3   na1 ,
• Lloyd Cool   ORCID: orcid.org/0000-0001-9936-3124 1 , 2 , 3 , 4 ,
• Beatriz Herrera-Malaver   ORCID: orcid.org/0000-0002-5096-9974 1 , 2 , 3 ,
• Christophe Vanderaa   ORCID: orcid.org/0000-0001-7443-5427 4 ,
• Florian A. Theßeling 1 , 2 , 3 ,
• Łukasz Kreft   ORCID: orcid.org/0000-0001-7620-4657 5 ,
• Alexander Botzki   ORCID: orcid.org/0000-0001-6691-4233 5 ,
• Philippe Malcorps 6 ,
• Luk Daenen 6 ,
• Tom Wenseleers   ORCID: orcid.org/0000-0002-1434-861X 4 &
• Kevin J. Verstrepen   ORCID: orcid.org/0000-0002-3077-6219 1 , 2 , 3  

Nature Communications volume  15 , Article number:  2368 ( 2024 ) Cite this article

49k Accesses

847 Altmetric

Metrics details

• Chemical engineering
• Gas chromatography
• Machine learning
• Metabolomics
• Taste receptors

The perception and appreciation of food flavor depends on many interacting chemical compounds and external factors, and therefore proves challenging to understand and predict. Here, we combine extensive chemical and sensory analyses of 250 different beers to train machine learning models that allow predicting flavor and consumer appreciation. For each beer, we measure over 200 chemical properties, perform quantitative descriptive sensory analysis with a trained tasting panel and map data from over 180,000 consumer reviews to train 10 different machine learning models. The best-performing algorithm, Gradient Boosting, yields models that significantly outperform predictions based on conventional statistics and accurately predict complex food features and consumer appreciation from chemical profiles. Model dissection allows identifying specific and unexpected compounds as drivers of beer flavor and appreciation. Adding these compounds results in variants of commercial alcoholic and non-alcoholic beers with improved consumer appreciation. Together, our study reveals how big data and machine learning uncover complex links between food chemistry, flavor and consumer perception, and lays the foundation to develop novel, tailored foods with superior flavors.

Similar content being viewed by others

Sensory lexicon and aroma volatiles analysis of brewing malt

Xiaoxia Su, Miao Yu, … Tianyi Du

Predicting odor from molecular structure: a multi-label classification approach

Kushagra Saini & Venkatnarayan Ramanathan

Toward a general and interpretable umami taste predictor using a multi-objective machine learning approach

Lorenzo Pallante, Aigli Korfiati, … Marco A. Deriu

Introduction

Predicting and understanding food perception and appreciation is one of the major challenges in food science. Accurate modeling of food flavor and appreciation could yield important opportunities for both producers and consumers, including quality control, product fingerprinting, counterfeit detection, spoilage detection, and the development of new products and product combinations (food pairing) 1 , 2 , 3 , 4 , 5 , 6 . Accurate models for flavor and consumer appreciation would contribute greatly to our scientific understanding of how humans perceive and appreciate flavor. Moreover, accurate predictive models would also facilitate and standardize existing food assessment methods and could supplement or replace assessments by trained and consumer tasting panels, which are variable, expensive and time-consuming 7 , 8 , 9 . Lastly, apart from providing objective, quantitative, accurate and contextual information that can help producers, models can also guide consumers in understanding their personal preferences 10 .

Despite the myriad of applications, predicting food flavor and appreciation from its chemical properties remains a largely elusive goal in sensory science, especially for complex food and beverages 11 , 12 . A key obstacle is the immense number of flavor-active chemicals underlying food flavor. Flavor compounds can vary widely in chemical structure and concentration, making them technically challenging and labor-intensive to quantify, even in the face of innovations in metabolomics, such as non-targeted metabolic fingerprinting 13 , 14 . Moreover, sensory analysis is perhaps even more complicated. Flavor perception is highly complex, resulting from hundreds of different molecules interacting at the physiochemical and sensorial level. Sensory perception is often non-linear, characterized by complex and concentration-dependent synergistic and antagonistic effects 15 , 16 , 17 , 18 , 19 , 20 , 21 that are further convoluted by the genetics, environment, culture and psychology of consumers 22 , 23 , 24 . Perceived flavor is therefore difficult to measure, with problems of sensitivity, accuracy, and reproducibility that can only be resolved by gathering sufficiently large datasets 25 . Trained tasting panels are considered the prime source of quality sensory data, but require meticulous training, are low throughput and high cost. Public databases containing consumer reviews of food products could provide a valuable alternative, especially for studying appreciation scores, which do not require formal training 25 . Public databases offer the advantage of amassing large amounts of data, increasing the statistical power to identify potential drivers of appreciation. However, public datasets suffer from biases, including a bias in the volunteers that contribute to the database, as well as confounding factors such as price, cult status and psychological conformity towards previous ratings of the product.

Classical multivariate statistics and machine learning methods have been used to predict flavor of specific compounds by, for example, linking structural properties of a compound to its potential biological activities or linking concentrations of specific compounds to sensory profiles 1 , 26 . Importantly, most previous studies focused on predicting organoleptic properties of single compounds (often based on their chemical structure) 27 , 28 , 29 , 30 , 31 , 32 , 33 , thus ignoring the fact that these compounds are present in a complex matrix in food or beverages and excluding complex interactions between compounds. Moreover, the classical statistics commonly used in sensory science 34 , 35 , 36 , 37 , 38 , 39 require a large sample size and sufficient variance amongst predictors to create accurate models. They are not fit for studying an extensive set of hundreds of interacting flavor compounds, since they are sensitive to outliers, have a high tendency to overfit and are less suited for non-linear and discontinuous relationships 40 .

In this study, we combine extensive chemical analyses and sensory data of a set of different commercial beers with machine learning approaches to develop models that predict taste, smell, mouthfeel and appreciation from compound concentrations. Beer is particularly suited to model the relationship between chemistry, flavor and appreciation. First, beer is a complex product, consisting of thousands of flavor compounds that partake in complex sensory interactions 41 , 42 , 43 . This chemical diversity arises from the raw materials (malt, yeast, hops, water and spices) and biochemical conversions during the brewing process (kilning, mashing, boiling, fermentation, maturation and aging) 44 , 45 . Second, the advent of the internet saw beer consumers embrace online review platforms, such as RateBeer (ZX Ventures, Anheuser-Busch InBev SA/NV) and BeerAdvocate (Next Glass, inc.). In this way, the beer community provides massive data sets of beer flavor and appreciation scores, creating extraordinarily large sensory databases to complement the analyses of our professional sensory panel. Specifically, we characterize over 200 chemical properties of 250 commercial beers, spread across 22 beer styles, and link these to the descriptive sensory profiling data of a 16-person in-house trained tasting panel and data acquired from over 180,000 public consumer reviews. These unique and extensive datasets enable us to train a suite of machine learning models to predict flavor and appreciation from a beer’s chemical profile. Dissection of the best-performing models allows us to pinpoint specific compounds as potential drivers of beer flavor and appreciation. Follow-up experiments confirm the importance of these compounds and ultimately allow us to significantly improve the flavor and appreciation of selected commercial beers. Together, our study represents a significant step towards understanding complex flavors and reinforces the value of machine learning to develop and refine complex foods. In this way, it represents a stepping stone for further computer-aided food engineering applications 46 .

To generate a comprehensive dataset on beer flavor, we selected 250 commercial Belgian beers across 22 different beer styles (Supplementary Fig.  S1 ). Beers with ≤ 4.2% alcohol by volume (ABV) were classified as non-alcoholic and low-alcoholic. Blonds and Tripels constitute a significant portion of the dataset (12.4% and 11.2%, respectively) reflecting their presence on the Belgian beer market and the heterogeneity of beers within these styles. By contrast, lager beers are less diverse and dominated by a handful of brands. Rare styles such as Brut or Faro make up only a small fraction of the dataset (2% and 1%, respectively) because fewer of these beers are produced and because they are dominated by distinct characteristics in terms of flavor and chemical composition.

Extensive analysis identifies relationships between chemical compounds in beer

For each beer, we measured 226 different chemical properties, including common brewing parameters such as alcohol content, iso-alpha acids, pH, sugar concentration 47 , and over 200 flavor compounds (Methods, Supplementary Table  S1 ). A large portion (37.2%) are terpenoids arising from hopping, responsible for herbal and fruity flavors 16 , 48 . A second major category are yeast metabolites, such as esters and alcohols, that result in fruity and solvent notes 48 , 49 , 50 . Other measured compounds are primarily derived from malt, or other microbes such as non- Saccharomyces yeasts and bacteria (‘wild flora’). Compounds that arise from spices or staling are labeled under ‘Others’. Five attributes (caloric value, total acids and total ester, hop aroma and sulfur compounds) are calculated from multiple individually measured compounds.

As a first step in identifying relationships between chemical properties, we determined correlations between the concentrations of the compounds (Fig.  1 , upper panel, Supplementary Data  1 and 2 , and Supplementary Fig.  S2 . For the sake of clarity, only a subset of the measured compounds is shown in Fig.  1 ). Compounds of the same origin typically show a positive correlation, while absence of correlation hints at parameters varying independently. For example, the hop aroma compounds citronellol, and alpha-terpineol show moderate correlations with each other (Spearman’s rho=0.39 and 0.57), but not with the bittering hop component iso-alpha acids (Spearman’s rho=0.16 and −0.07). This illustrates how brewers can independently modify hop aroma and bitterness by selecting hop varieties and dosage time. If hops are added early in the boiling phase, chemical conversions increase bitterness while aromas evaporate, conversely, late addition of hops preserves aroma but limits bitterness 51 . Similarly, hop-derived iso-alpha acids show a strong anti-correlation with lactic acid and acetic acid, likely reflecting growth inhibition of lactic acid and acetic acid bacteria, or the consequent use of fewer hops in sour beer styles, such as West Flanders ales and Fruit beers, that rely on these bacteria for their distinct flavors 52 . Finally, yeast-derived esters (ethyl acetate, ethyl decanoate, ethyl hexanoate, ethyl octanoate) and alcohols (ethanol, isoamyl alcohol, isobutanol, and glycerol), correlate with Spearman coefficients above 0.5, suggesting that these secondary metabolites are correlated with the yeast genetic background and/or fermentation parameters and may be difficult to influence individually, although the choice of yeast strain may offer some control 53 .

Spearman rank correlations are shown. Descriptors are grouped according to their origin (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)), and sensory aspect (aroma, taste, palate, and overall appreciation). Please note that for the chemical compounds, for the sake of clarity, only a subset of the total number of measured compounds is shown, with an emphasis on the key compounds for each source. For more details, see the main text and Methods section. Chemical data can be found in Supplementary Data  1 , correlations between all chemical compounds are depicted in Supplementary Fig.  S2 and correlation values can be found in Supplementary Data  2 . See Supplementary Data  4 for sensory panel assessments and Supplementary Data  5 for correlation values between all sensory descriptors.

Interestingly, different beer styles show distinct patterns for some flavor compounds (Supplementary Fig.  S3 ). These observations agree with expectations for key beer styles, and serve as a control for our measurements. For instance, Stouts generally show high values for color (darker), while hoppy beers contain elevated levels of iso-alpha acids, compounds associated with bitter hop taste. Acetic and lactic acid are not prevalent in most beers, with notable exceptions such as Kriek, Lambic, Faro, West Flanders ales and Flanders Old Brown, which use acid-producing bacteria ( Lactobacillus and Pediococcus ) or unconventional yeast ( Brettanomyces ) 54 , 55 . Glycerol, ethanol and esters show similar distributions across all beer styles, reflecting their common origin as products of yeast metabolism during fermentation 45 , 53 . Finally, low/no-alcohol beers contain low concentrations of glycerol and esters. This is in line with the production process for most of the low/no-alcohol beers in our dataset, which are produced through limiting fermentation or by stripping away alcohol via evaporation or dialysis, with both methods having the unintended side-effect of reducing the amount of flavor compounds in the final beer 56 , 57 .

Besides expected associations, our data also reveals less trivial associations between beer styles and specific parameters. For example, geraniol and citronellol, two monoterpenoids responsible for citrus, floral and rose flavors and characteristic of Citra hops, are found in relatively high amounts in Christmas, Saison, and Brett/co-fermented beers, where they may originate from terpenoid-rich spices such as coriander seeds instead of hops 58 .

Tasting panel assessments reveal sensorial relationships in beer

To assess the sensory profile of each beer, a trained tasting panel evaluated each of the 250 beers for 50 sensory attributes, including different hop, malt and yeast flavors, off-flavors and spices. Panelists used a tasting sheet (Supplementary Data  3 ) to score the different attributes. Panel consistency was evaluated by repeating 12 samples across different sessions and performing ANOVA. In 95% of cases no significant difference was found across sessions ( p  > 0.05), indicating good panel consistency (Supplementary Table  S2 ).

Aroma and taste perception reported by the trained panel are often linked (Fig.  1 , bottom left panel and Supplementary Data  4 and 5 ), with high correlations between hops aroma and taste (Spearman’s rho=0.83). Bitter taste was found to correlate with hop aroma and taste in general (Spearman’s rho=0.80 and 0.69), and particularly with “grassy” noble hops (Spearman’s rho=0.75). Barnyard flavor, most often associated with sour beers, is identified together with stale hops (Spearman’s rho=0.97) that are used in these beers. Lactic and acetic acid, which often co-occur, are correlated (Spearman’s rho=0.66). Interestingly, sweetness and bitterness are anti-correlated (Spearman’s rho = −0.48), confirming the hypothesis that they mask each other 59 , 60 . Beer body is highly correlated with alcohol (Spearman’s rho = 0.79), and overall appreciation is found to correlate with multiple aspects that describe beer mouthfeel (alcohol, carbonation; Spearman’s rho= 0.32, 0.39), as well as with hop and ester aroma intensity (Spearman’s rho=0.39 and 0.35).

Similar to the chemical analyses, sensorial analyses confirmed typical features of specific beer styles (Supplementary Fig.  S4 ). For example, sour beers (Faro, Flanders Old Brown, Fruit beer, Kriek, Lambic, West Flanders ale) were rated acidic, with flavors of both acetic and lactic acid. Hoppy beers were found to be bitter and showed hop-associated aromas like citrus and tropical fruit. Malt taste is most detected among scotch, stout/porters, and strong ales, while low/no-alcohol beers, which often have a reputation for being ‘worty’ (reminiscent of unfermented, sweet malt extract) appear in the middle. Unsurprisingly, hop aromas are most strongly detected among hoppy beers. Like its chemical counterpart (Supplementary Fig.  S3 ), acidity shows a right-skewed distribution, with the most acidic beers being Krieks, Lambics, and West Flanders ales.

Tasting panel assessments of specific flavors correlate with chemical composition

We find that the concentrations of several chemical compounds strongly correlate with specific aroma or taste, as evaluated by the tasting panel (Fig.  2 , Supplementary Fig.  S5 , Supplementary Data  6 ). In some cases, these correlations confirm expectations and serve as a useful control for data quality. For example, iso-alpha acids, the bittering compounds in hops, strongly correlate with bitterness (Spearman’s rho=0.68), while ethanol and glycerol correlate with tasters’ perceptions of alcohol and body, the mouthfeel sensation of fullness (Spearman’s rho=0.82/0.62 and 0.72/0.57 respectively) and darker color from roasted malts is a good indication of malt perception (Spearman’s rho=0.54).

Heatmap colors indicate Spearman’s Rho. Axes are organized according to sensory categories (aroma, taste, mouthfeel, overall), chemical categories and chemical sources in beer (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)). See Supplementary Data  6 for all correlation values.

Interestingly, for some relationships between chemical compounds and perceived flavor, correlations are weaker than expected. For example, the rose-smelling phenethyl acetate only weakly correlates with floral aroma. This hints at more complex relationships and interactions between compounds and suggests a need for a more complex model than simple correlations. Lastly, we uncovered unexpected correlations. For instance, the esters ethyl decanoate and ethyl octanoate appear to correlate slightly with hop perception and bitterness, possibly due to their fruity flavor. Iron is anti-correlated with hop aromas and bitterness, most likely because it is also anti-correlated with iso-alpha acids. This could be a sign of metal chelation of hop acids 61 , given that our analyses measure unbound hop acids and total iron content, or could result from the higher iron content in dark and Fruit beers, which typically have less hoppy and bitter flavors 62 .

Public consumer reviews complement expert panel data

To complement and expand the sensory data of our trained tasting panel, we collected 180,000 reviews of our 250 beers from the online consumer review platform RateBeer. This provided numerical scores for beer appearance, aroma, taste, palate, overall quality as well as the average overall score.

Public datasets are known to suffer from biases, such as price, cult status and psychological conformity towards previous ratings of a product. For example, prices correlate with appreciation scores for these online consumer reviews (rho=0.49, Supplementary Fig.  S6 ), but not for our trained tasting panel (rho=0.19). This suggests that prices affect consumer appreciation, which has been reported in wine 63 , while blind tastings are unaffected. Moreover, we observe that some beer styles, like lagers and non-alcoholic beers, generally receive lower scores, reflecting that online reviewers are mostly beer aficionados with a preference for specialty beers over lager beers. In general, we find a modest correlation between our trained panel’s overall appreciation score and the online consumer appreciation scores (Fig.  3 , rho=0.29). Apart from the aforementioned biases in the online datasets, serving temperature, sample freshness and surroundings, which are all tightly controlled during the tasting panel sessions, can vary tremendously across online consumers and can further contribute to (among others, appreciation) differences between the two categories of tasters. Importantly, in contrast to the overall appreciation scores, for many sensory aspects the results from the professional panel correlated well with results obtained from RateBeer reviews. Correlations were highest for features that are relatively easy to recognize even for untrained tasters, like bitterness, sweetness, alcohol and malt aroma (Fig.  3 and below).

RateBeer text mining results can be found in Supplementary Data  7 . Rho values shown are Spearman correlation values, with asterisks indicating significant correlations ( p  < 0.05, two-sided). All p values were smaller than 0.001, except for Esters aroma (0.0553), Esters taste (0.3275), Esters aroma—banana (0.0019), Coriander (0.0508) and Diacetyl (0.0134).

Besides collecting consumer appreciation from these online reviews, we developed automated text analysis tools to gather additional data from review texts (Supplementary Data  7 ). Processing review texts on the RateBeer database yielded comparable results to the scores given by the trained panel for many common sensory aspects, including acidity, bitterness, sweetness, alcohol, malt, and hop tastes (Fig.  3 ). This is in line with what would be expected, since these attributes require less training for accurate assessment and are less influenced by environmental factors such as temperature, serving glass and odors in the environment. Consumer reviews also correlate well with our trained panel for 4-vinyl guaiacol, a compound associated with a very characteristic aroma. By contrast, correlations for more specific aromas like ester, coriander or diacetyl are underrepresented in the online reviews, underscoring the importance of using a trained tasting panel and standardized tasting sheets with explicit factors to be scored for evaluating specific aspects of a beer. Taken together, our results suggest that public reviews are trustworthy for some, but not all, flavor features and can complement or substitute taste panel data for these sensory aspects.

Models can predict beer sensory profiles from chemical data

The rich datasets of chemical analyses, tasting panel assessments and public reviews gathered in the first part of this study provided us with a unique opportunity to develop predictive models that link chemical data to sensorial features. Given the complexity of beer flavor, basic statistical tools such as correlations or linear regression may not always be the most suitable for making accurate predictions. Instead, we applied different machine learning models that can model both simple linear and complex interactive relationships. Specifically, we constructed a set of regression models to predict (a) trained panel scores for beer flavor and quality and (b) public reviews’ appreciation scores from beer chemical profiles. We trained and tested 10 different models (Methods), 3 linear regression-based models (simple linear regression with first-order interactions (LR), lasso regression with first-order interactions (Lasso), partial least squares regressor (PLSR)), 5 decision tree models (AdaBoost regressor (ABR), extra trees (ET), gradient boosting regressor (GBR), random forest (RF) and XGBoost regressor (XGBR)), 1 support vector regression (SVR), and 1 artificial neural network (ANN) model.

To compare the performance of our machine learning models, the dataset was randomly split into a training and test set, stratified by beer style. After a model was trained on data in the training set, its performance was evaluated on its ability to predict the test dataset obtained from multi-output models (based on the coefficient of determination, see Methods). Additionally, individual-attribute models were ranked per descriptor and the average rank was calculated, as proposed by Korneva et al. 64 . Importantly, both ways of evaluating the models’ performance agreed in general. Performance of the different models varied (Table  1 ). It should be noted that all models perform better at predicting RateBeer results than results from our trained tasting panel. One reason could be that sensory data is inherently variable, and this variability is averaged out with the large number of public reviews from RateBeer. Additionally, all tree-based models perform better at predicting taste than aroma. Linear models (LR) performed particularly poorly, with negative R 2 values, due to severe overfitting (training set R 2  = 1). Overfitting is a common issue in linear models with many parameters and limited samples, especially with interaction terms further amplifying the number of parameters. L1 regularization (Lasso) successfully overcomes this overfitting, out-competing multiple tree-based models on the RateBeer dataset. Similarly, the dimensionality reduction of PLSR avoids overfitting and improves performance, to some extent. Still, tree-based models (ABR, ET, GBR, RF and XGBR) show the best performance, out-competing the linear models (LR, Lasso, PLSR) commonly used in sensory science 65 .

GBR models showed the best overall performance in predicting sensory responses from chemical information, with R 2 values up to 0.75 depending on the predicted sensory feature (Supplementary Table  S4 ). The GBR models predict consumer appreciation (RateBeer) better than our trained panel’s appreciation (R 2 value of 0.67 compared to R 2 value of 0.09) (Supplementary Table  S3 and Supplementary Table  S4 ). ANN models showed intermediate performance, likely because neural networks typically perform best with larger datasets 66 . The SVR shows intermediate performance, mostly due to the weak predictions of specific attributes that lower the overall performance (Supplementary Table  S4 ).

Model dissection identifies specific, unexpected compounds as drivers of consumer appreciation

Next, we leveraged our models to infer important contributors to sensory perception and consumer appreciation. Consumer preference is a crucial sensory aspects, because a product that shows low consumer appreciation scores often does not succeed commercially 25 . Additionally, the requirement for a large number of representative evaluators makes consumer trials one of the more costly and time-consuming aspects of product development. Hence, a model for predicting chemical drivers of overall appreciation would be a welcome addition to the available toolbox for food development and optimization.

Since GBR models on our RateBeer dataset showed the best overall performance, we focused on these models. Specifically, we used two approaches to identify important contributors. First, rankings of the most important predictors for each sensorial trait in the GBR models were obtained based on impurity-based feature importance (mean decrease in impurity). High-ranked parameters were hypothesized to be either the true causal chemical properties underlying the trait, to correlate with the actual causal properties, or to take part in sensory interactions affecting the trait 67 (Fig.  4A ). In a second approach, we used SHAP 68 to determine which parameters contributed most to the model for making predictions of consumer appreciation (Fig.  4B ). SHAP calculates parameter contributions to model predictions on a per-sample basis, which can be aggregated into an importance score.

A The impurity-based feature importance (mean deviance in impurity, MDI) calculated from the Gradient Boosting Regression (GBR) model predicting RateBeer appreciation scores. The top 15 highest ranked chemical properties are shown. B SHAP summary plot for the top 15 parameters contributing to our GBR model. Each point on the graph represents a sample from our dataset. The color represents the concentration of that parameter, with bluer colors representing low values and redder colors representing higher values. Greater absolute values on the horizontal axis indicate a higher impact of the parameter on the prediction of the model. C Spearman correlations between the 15 most important chemical properties and consumer overall appreciation. Numbers indicate the Spearman Rho correlation coefficient, and the rank of this correlation compared to all other correlations. The top 15 important compounds were determined using SHAP (panel B).

Both approaches identified ethyl acetate as the most predictive parameter for beer appreciation (Fig.  4 ). Ethyl acetate is the most abundant ester in beer with a typical ‘fruity’, ‘solvent’ and ‘alcoholic’ flavor, but is often considered less important than other esters like isoamyl acetate. The second most important parameter identified by SHAP is ethanol, the most abundant beer compound after water. Apart from directly contributing to beer flavor and mouthfeel, ethanol drastically influences the physical properties of beer, dictating how easily volatile compounds escape the beer matrix to contribute to beer aroma 69 . Importantly, it should also be noted that the importance of ethanol for appreciation is likely inflated by the very low appreciation scores of non-alcoholic beers (Supplementary Fig.  S4 ). Despite not often being considered a driver of beer appreciation, protein level also ranks highly in both approaches, possibly due to its effect on mouthfeel and body 70 . Lactic acid, which contributes to the tart taste of sour beers, is the fourth most important parameter identified by SHAP, possibly due to the generally high appreciation of sour beers in our dataset.

Interestingly, some of the most important predictive parameters for our model are not well-established as beer flavors or are even commonly regarded as being negative for beer quality. For example, our models identify methanethiol and ethyl phenyl acetate, an ester commonly linked to beer staling 71 , as a key factor contributing to beer appreciation. Although there is no doubt that high concentrations of these compounds are considered unpleasant, the positive effects of modest concentrations are not yet known 72 , 73 .

To compare our approach to conventional statistics, we evaluated how well the 15 most important SHAP-derived parameters correlate with consumer appreciation (Fig.  4C ). Interestingly, only 6 of the properties derived by SHAP rank amongst the top 15 most correlated parameters. For some chemical compounds, the correlations are so low that they would have likely been considered unimportant. For example, lactic acid, the fourth most important parameter, shows a bimodal distribution for appreciation, with sour beers forming a separate cluster, that is missed entirely by the Spearman correlation. Additionally, the correlation plots reveal outliers, emphasizing the need for robust analysis tools. Together, this highlights the need for alternative models, like the Gradient Boosting model, that better grasp the complexity of (beer) flavor.

Finally, to observe the relationships between these chemical properties and their predicted targets, partial dependence plots were constructed for the six most important predictors of consumer appreciation 74 , 75 , 76 (Supplementary Fig.  S7 ). One-way partial dependence plots show how a change in concentration affects the predicted appreciation. These plots reveal an important limitation of our models: appreciation predictions remain constant at ever-increasing concentrations. This implies that once a threshold concentration is reached, further increasing the concentration does not affect appreciation. This is false, as it is well-documented that certain compounds become unpleasant at high concentrations, including ethyl acetate (‘nail polish’) 77 and methanethiol (‘sulfury’ and ‘rotten cabbage’) 78 . The inability of our models to grasp that flavor compounds have optimal levels, above which they become negative, is a consequence of working with commercial beer brands where (off-)flavors are rarely too high to negatively impact the product. The two-way partial dependence plots show how changing the concentration of two compounds influences predicted appreciation, visualizing their interactions (Supplementary Fig.  S7 ). In our case, the top 5 parameters are dominated by additive or synergistic interactions, with high concentrations for both compounds resulting in the highest predicted appreciation.

To assess the robustness of our best-performing models and model predictions, we performed 100 iterations of the GBR, RF and ET models. In general, all iterations of the models yielded similar performance (Supplementary Fig.  S8 ). Moreover, the main predictors (including the top predictors ethanol and ethyl acetate) remained virtually the same, especially for GBR and RF. For the iterations of the ET model, we did observe more variation in the top predictors, which is likely a consequence of the model’s inherent random architecture in combination with co-correlations between certain predictors. However, even in this case, several of the top predictors (ethanol and ethyl acetate) remain unchanged, although their rank in importance changes (Supplementary Fig.  S8 ).

Next, we investigated if a combination of RateBeer and trained panel data into one consolidated dataset would lead to stronger models, under the hypothesis that such a model would suffer less from bias in the datasets. A GBR model was trained to predict appreciation on the combined dataset. This model underperformed compared to the RateBeer model, both in the native case and when including a dataset identifier (R 2  = 0.67, 0.26 and 0.42 respectively). For the latter, the dataset identifier is the most important feature (Supplementary Fig.  S9 ), while most of the feature importance remains unchanged, with ethyl acetate and ethanol ranking highest, like in the original model trained only on RateBeer data. It seems that the large variation in the panel dataset introduces noise, weakening the models’ performances and reliability. In addition, it seems reasonable to assume that both datasets are fundamentally different, with the panel dataset obtained by blind tastings by a trained professional panel.

Lastly, we evaluated whether beer style identifiers would further enhance the model’s performance. A GBR model was trained with parameters that explicitly encoded the styles of the samples. This did not improve model performance (R2 = 0.66 with style information vs R2 = 0.67). The most important chemical features are consistent with the model trained without style information (eg. ethanol and ethyl acetate), and with the exception of the most preferred (strong ale) and least preferred (low/no-alcohol) styles, none of the styles were among the most important features (Supplementary Fig.  S9 , Supplementary Table  S5 and S6 ). This is likely due to a combination of style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original models, as well as the low number of samples belonging to some styles, making it difficult for the model to learn style-specific patterns. Moreover, beer styles are not rigorously defined, with some styles overlapping in features and some beers being misattributed to a specific style, all of which leads to more noise in models that use style parameters.

Model validation

To test if our predictive models give insight into beer appreciation, we set up experiments aimed at improving existing commercial beers. We specifically selected overall appreciation as the trait to be examined because of its complexity and commercial relevance. Beer flavor comprises a complex bouquet rather than single aromas and tastes 53 . Hence, adding a single compound to the extent that a difference is noticeable may lead to an unbalanced, artificial flavor. Therefore, we evaluated the effect of combinations of compounds. Because Blond beers represent the most extensive style in our dataset, we selected a beer from this style as the starting material for these experiments (Beer 64 in Supplementary Data  1 ).

In the first set of experiments, we adjusted the concentrations of compounds that made up the most important predictors of overall appreciation (ethyl acetate, ethanol, lactic acid, ethyl phenyl acetate) together with correlated compounds (ethyl hexanoate, isoamyl acetate, glycerol), bringing them up to 95 th percentile ethanol-normalized concentrations (Methods) within the Blond group (‘Spiked’ concentration in Fig.  5A ). Compared to controls, the spiked beers were found to have significantly improved overall appreciation among trained panelists, with panelist noting increased intensity of ester flavors, sweetness, alcohol, and body fullness (Fig.  5B ). To disentangle the contribution of ethanol to these results, a second experiment was performed without the addition of ethanol. This resulted in a similar outcome, including increased perception of alcohol and overall appreciation.

Adding the top chemical compounds, identified as best predictors of appreciation by our model, into poorly appreciated beers results in increased appreciation from our trained panel. Results of sensory tests between base beers and those spiked with compounds identified as the best predictors by the model. A Blond and Non/Low-alcohol (0.0% ABV) base beers were brought up to 95th-percentile ethanol-normalized concentrations within each style. B For each sensory attribute, tasters indicated the more intense sample and selected the sample they preferred. The numbers above the bars correspond to the p values that indicate significant changes in perceived flavor (two-sided binomial test: alpha 0.05, n  = 20 or 13).

In a last experiment, we tested whether using the model’s predictions can boost the appreciation of a non-alcoholic beer (beer 223 in Supplementary Data  1 ). Again, the addition of a mixture of predicted compounds (omitting ethanol, in this case) resulted in a significant increase in appreciation, body, ester flavor and sweetness.

Predicting flavor and consumer appreciation from chemical composition is one of the ultimate goals of sensory science. A reliable, systematic and unbiased way to link chemical profiles to flavor and food appreciation would be a significant asset to the food and beverage industry. Such tools would substantially aid in quality control and recipe development, offer an efficient and cost-effective alternative to pilot studies and consumer trials and would ultimately allow food manufacturers to produce superior, tailor-made products that better meet the demands of specific consumer groups more efficiently.

A limited set of studies have previously tried, to varying degrees of success, to predict beer flavor and beer popularity based on (a limited set of) chemical compounds and flavors 79 , 80 . Current sensitive, high-throughput technologies allow measuring an unprecedented number of chemical compounds and properties in a large set of samples, yielding a dataset that can train models that help close the gaps between chemistry and flavor, even for a complex natural product like beer. To our knowledge, no previous research gathered data at this scale (250 samples, 226 chemical parameters, 50 sensory attributes and 5 consumer scores) to disentangle and validate the chemical aspects driving beer preference using various machine-learning techniques. We find that modern machine learning models outperform conventional statistical tools, such as correlations and linear models, and can successfully predict flavor appreciation from chemical composition. This could be attributed to the natural incorporation of interactions and non-linear or discontinuous effects in machine learning models, which are not easily grasped by the linear model architecture. While linear models and partial least squares regression represent the most widespread statistical approaches in sensory science, in part because they allow interpretation 65 , 81 , 82 , modern machine learning methods allow for building better predictive models while preserving the possibility to dissect and exploit the underlying patterns. Of the 10 different models we trained, tree-based models, such as our best performing GBR, showed the best overall performance in predicting sensory responses from chemical information, outcompeting artificial neural networks. This agrees with previous reports for models trained on tabular data 83 . Our results are in line with the findings of Colantonio et al. who also identified the gradient boosting architecture as performing best at predicting appreciation and flavor (of tomatoes and blueberries, in their specific study) 26 . Importantly, besides our larger experimental scale, we were able to directly confirm our models’ predictions in vivo.

Our study confirms that flavor compound concentration does not always correlate with perception, suggesting complex interactions that are often missed by more conventional statistics and simple models. Specifically, we find that tree-based algorithms may perform best in developing models that link complex food chemistry with aroma. Furthermore, we show that massive datasets of untrained consumer reviews provide a valuable source of data, that can complement or even replace trained tasting panels, especially for appreciation and basic flavors, such as sweetness and bitterness. This holds despite biases that are known to occur in such datasets, such as price or conformity bias. Moreover, GBR models predict taste better than aroma. This is likely because taste (e.g. bitterness) often directly relates to the corresponding chemical measurements (e.g., iso-alpha acids), whereas such a link is less clear for aromas, which often result from the interplay between multiple volatile compounds. We also find that our models are best at predicting acidity and alcohol, likely because there is a direct relation between the measured chemical compounds (acids and ethanol) and the corresponding perceived sensorial attribute (acidity and alcohol), and because even untrained consumers are generally able to recognize these flavors and aromas.

The predictions of our final models, trained on review data, hold even for blind tastings with small groups of trained tasters, as demonstrated by our ability to validate specific compounds as drivers of beer flavor and appreciation. Since adding a single compound to the extent of a noticeable difference may result in an unbalanced flavor profile, we specifically tested our identified key drivers as a combination of compounds. While this approach does not allow us to validate if a particular single compound would affect flavor and/or appreciation, our experiments do show that this combination of compounds increases consumer appreciation.

It is important to stress that, while it represents an important step forward, our approach still has several major limitations. A key weakness of the GBR model architecture is that amongst co-correlating variables, the largest main effect is consistently preferred for model building. As a result, co-correlating variables often have artificially low importance scores, both for impurity and SHAP-based methods, like we observed in the comparison to the more randomized Extra Trees models. This implies that chemicals identified as key drivers of a specific sensory feature by GBR might not be the true causative compounds, but rather co-correlate with the actual causative chemical. For example, the high importance of ethyl acetate could be (partially) attributed to the total ester content, ethanol or ethyl hexanoate (rho=0.77, rho=0.72 and rho=0.68), while ethyl phenylacetate could hide the importance of prenyl isobutyrate and ethyl benzoate (rho=0.77 and rho=0.76). Expanding our GBR model to include beer style as a parameter did not yield additional power or insight. This is likely due to style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original model, as well as the smaller sample size per style, limiting the power to uncover style-specific patterns. This can be partly attributed to the curse of dimensionality, where the high number of parameters results in the models mainly incorporating single parameter effects, rather than complex interactions such as style-dependent effects 67 . A larger number of samples may overcome some of these limitations and offer more insight into style-specific effects. On the other hand, beer style is not a rigid scientific classification, and beers within one style often differ a lot, which further complicates the analysis of style as a model factor.

Our study is limited to beers from Belgian breweries. Although these beers cover a large portion of the beer styles available globally, some beer styles and consumer patterns may be missing, while other features might be overrepresented. For example, many Belgian ales exhibit yeast-driven flavor profiles, which is reflected in the chemical drivers of appreciation discovered by this study. In future work, expanding the scope to include diverse markets and beer styles could lead to the identification of even more drivers of appreciation and better models for special niche products that were not present in our beer set.

In addition to inherent limitations of GBR models, there are also some limitations associated with studying food aroma. Even if our chemical analyses measured most of the known aroma compounds, the total number of flavor compounds in complex foods like beer is still larger than the subset we were able to measure in this study. For example, hop-derived thiols, that influence flavor at very low concentrations, are notoriously difficult to measure in a high-throughput experiment. Moreover, consumer perception remains subjective and prone to biases that are difficult to avoid. It is also important to stress that the models are still immature and that more extensive datasets will be crucial for developing more complete models in the future. Besides more samples and parameters, our dataset does not include any demographic information about the tasters. Including such data could lead to better models that grasp external factors like age and culture. Another limitation is that our set of beers consists of high-quality end-products and lacks beers that are unfit for sale, which limits the current model in accurately predicting products that are appreciated very badly. Finally, while models could be readily applied in quality control, their use in sensory science and product development is restrained by their inability to discern causal relationships. Given that the models cannot distinguish compounds that genuinely drive consumer perception from those that merely correlate, validation experiments are essential to identify true causative compounds.

Despite the inherent limitations, dissection of our models enabled us to pinpoint specific molecules as potential drivers of beer aroma and consumer appreciation, including compounds that were unexpected and would not have been identified using standard approaches. Important drivers of beer appreciation uncovered by our models include protein levels, ethyl acetate, ethyl phenyl acetate and lactic acid. Currently, many brewers already use lactic acid to acidify their brewing water and ensure optimal pH for enzymatic activity during the mashing process. Our results suggest that adding lactic acid can also improve beer appreciation, although its individual effect remains to be tested. Interestingly, ethanol appears to be unnecessary to improve beer appreciation, both for blond beer and alcohol-free beer. Given the growing consumer interest in alcohol-free beer, with a predicted annual market growth of >7% 84 , it is relevant for brewers to know what compounds can further increase consumer appreciation of these beers. Hence, our model may readily provide avenues to further improve the flavor and consumer appreciation of both alcoholic and non-alcoholic beers, which is generally considered one of the key challenges for future beer production.

Whereas we see a direct implementation of our results for the development of superior alcohol-free beverages and other food products, our study can also serve as a stepping stone for the development of novel alcohol-containing beverages. We want to echo the growing body of scientific evidence for the negative effects of alcohol consumption, both on the individual level by the mutagenic, teratogenic and carcinogenic effects of ethanol 85 , 86 , as well as the burden on society caused by alcohol abuse and addiction. We encourage the use of our results for the production of healthier, tastier products, including novel and improved beverages with lower alcohol contents. Furthermore, we strongly discourage the use of these technologies to improve the appreciation or addictive properties of harmful substances.

The present work demonstrates that despite some important remaining hurdles, combining the latest developments in chemical analyses, sensory analysis and modern machine learning methods offers exciting avenues for food chemistry and engineering. Soon, these tools may provide solutions in quality control and recipe development, as well as new approaches to sensory science and flavor research.

Beer selection

250 commercial Belgian beers were selected to cover the broad diversity of beer styles and corresponding diversity in chemical composition and aroma. See Supplementary Fig.  S1 .

Chemical dataset

Sample preparation.

Beers within their expiration date were purchased from commercial retailers. Samples were prepared in biological duplicates at room temperature, unless explicitly stated otherwise. Bottle pressure was measured with a manual pressure device (Steinfurth Mess-Systeme GmbH) and used to calculate CO 2 concentration. The beer was poured through two filter papers (Macherey-Nagel, 500713032 MN 713 ¼) to remove carbon dioxide and prevent spontaneous foaming. Samples were then prepared for measurements by targeted Headspace-Gas Chromatography-Flame Ionization Detector/Flame Photometric Detector (HS-GC-FID/FPD), Headspace-Solid Phase Microextraction-Gas Chromatography-Mass Spectrometry (HS-SPME-GC-MS), colorimetric analysis, enzymatic analysis, Near-Infrared (NIR) analysis, as described in the sections below. The mean values of biological duplicates are reported for each compound.

HS-GC-FID/FPD

HS-GC-FID/FPD (Shimadzu GC 2010 Plus) was used to measure higher alcohols, acetaldehyde, esters, 4-vinyl guaicol, and sulfur compounds. Each measurement comprised 5 ml of sample pipetted into a 20 ml glass vial containing 1.75 g NaCl (VWR, 27810.295). 100 µl of 2-heptanol (Sigma-Aldrich, H3003) (internal standard) solution in ethanol (Fisher Chemical, E/0650DF/C17) was added for a final concentration of 2.44 mg/L. Samples were flushed with nitrogen for 10 s, sealed with a silicone septum, stored at −80 °C and analyzed in batches of 20.

The GC was equipped with a DB-WAXetr column (length, 30 m; internal diameter, 0.32 mm; layer thickness, 0.50 µm; Agilent Technologies, Santa Clara, CA, USA) to the FID and an HP-5 column (length, 30 m; internal diameter, 0.25 mm; layer thickness, 0.25 µm; Agilent Technologies, Santa Clara, CA, USA) to the FPD. N 2 was used as the carrier gas. Samples were incubated for 20 min at 70 °C in the headspace autosampler (Flow rate, 35 cm/s; Injection volume, 1000 µL; Injection mode, split; Combi PAL autosampler, CTC analytics, Switzerland). The injector, FID and FPD temperatures were kept at 250 °C. The GC oven temperature was first held at 50 °C for 5 min and then allowed to rise to 80 °C at a rate of 5 °C/min, followed by a second ramp of 4 °C/min until 200 °C kept for 3 min and a final ramp of (4 °C/min) until 230 °C for 1 min. Results were analyzed with the GCSolution software version 2.4 (Shimadzu, Kyoto, Japan). The GC was calibrated with a 5% EtOH solution (VWR International) containing the volatiles under study (Supplementary Table  S7 ).

HS-SPME-GC-MS

HS-SPME-GC-MS (Shimadzu GCMS-QP-2010 Ultra) was used to measure additional volatile compounds, mainly comprising terpenoids and esters. Samples were analyzed by HS-SPME using a triphase DVB/Carboxen/PDMS 50/30 μm SPME fiber (Supelco Co., Bellefonte, PA, USA) followed by gas chromatography (Thermo Fisher Scientific Trace 1300 series, USA) coupled to a mass spectrometer (Thermo Fisher Scientific ISQ series MS) equipped with a TriPlus RSH autosampler. 5 ml of degassed beer sample was placed in 20 ml vials containing 1.75 g NaCl (VWR, 27810.295). 5 µl internal standard mix was added, containing 2-heptanol (1 g/L) (Sigma-Aldrich, H3003), 4-fluorobenzaldehyde (1 g/L) (Sigma-Aldrich, 128376), 2,3-hexanedione (1 g/L) (Sigma-Aldrich, 144169) and guaiacol (1 g/L) (Sigma-Aldrich, W253200) in ethanol (Fisher Chemical, E/0650DF/C17). Each sample was incubated at 60 °C in the autosampler oven with constant agitation. After 5 min equilibration, the SPME fiber was exposed to the sample headspace for 30 min. The compounds trapped on the fiber were thermally desorbed in the injection port of the chromatograph by heating the fiber for 15 min at 270 °C.

The GC-MS was equipped with a low polarity RXi-5Sil MS column (length, 20 m; internal diameter, 0.18 mm; layer thickness, 0.18 µm; Restek, Bellefonte, PA, USA). Injection was performed in splitless mode at 320 °C, a split flow of 9 ml/min, a purge flow of 5 ml/min and an open valve time of 3 min. To obtain a pulsed injection, a programmed gas flow was used whereby the helium gas flow was set at 2.7 mL/min for 0.1 min, followed by a decrease in flow of 20 ml/min to the normal 0.9 mL/min. The temperature was first held at 30 °C for 3 min and then allowed to rise to 80 °C at a rate of 7 °C/min, followed by a second ramp of 2 °C/min till 125 °C and a final ramp of 8 °C/min with a final temperature of 270 °C.

Mass acquisition range was 33 to 550 amu at a scan rate of 5 scans/s. Electron impact ionization energy was 70 eV. The interface and ion source were kept at 275 °C and 250 °C, respectively. A mix of linear n-alkanes (from C7 to C40, Supelco Co.) was injected into the GC-MS under identical conditions to serve as external retention index markers. Identification and quantification of the compounds were performed using an in-house developed R script as described in Goelen et al. and Reher et al. 87 , 88 (for package information, see Supplementary Table  S8 ). Briefly, chromatograms were analyzed using AMDIS (v2.71) 89 to separate overlapping peaks and obtain pure compound spectra. The NIST MS Search software (v2.0 g) in combination with the NIST2017, FFNSC3 and Adams4 libraries were used to manually identify the empirical spectra, taking into account the expected retention time. After background subtraction and correcting for retention time shifts between samples run on different days based on alkane ladders, compound elution profiles were extracted and integrated using a file with 284 target compounds of interest, which were either recovered in our identified AMDIS list of spectra or were known to occur in beer. Compound elution profiles were estimated for every peak in every chromatogram over a time-restricted window using weighted non-negative least square analysis after which peak areas were integrated 87 , 88 . Batch effect correction was performed by normalizing against the most stable internal standard compound, 4-fluorobenzaldehyde. Out of all 284 target compounds that were analyzed, 167 were visually judged to have reliable elution profiles and were used for final analysis.

Discrete photometric and enzymatic analysis

Discrete photometric and enzymatic analysis (Thermo Scientific TM Gallery TM Plus Beermaster Discrete Analyzer) was used to measure acetic acid, ammonia, beta-glucan, iso-alpha acids, color, sugars, glycerol, iron, pH, protein, and sulfite. 2 ml of sample volume was used for the analyses. Information regarding the reagents and standard solutions used for analyses and calibrations is included in Supplementary Table  S7 and Supplementary Table  S9 .

NIR analyses

NIR analysis (Anton Paar Alcolyzer Beer ME System) was used to measure ethanol. Measurements comprised 50 ml of sample, and a 10% EtOH solution was used for calibration.

Correlation calculations

Pairwise Spearman Rank correlations were calculated between all chemical properties.

Sensory dataset

Trained panel.

Our trained tasting panel consisted of volunteers who gave prior verbal informed consent. All compounds used for the validation experiment were of food-grade quality. The tasting sessions were approved by the Social and Societal Ethics Committee of the KU Leuven (G-2022-5677-R2(MAR)). All online reviewers agreed to the Terms and Conditions of the RateBeer website.

Sensory analysis was performed according to the American Society of Brewing Chemists (ASBC) Sensory Analysis Methods 90 . 30 volunteers were screened through a series of triangle tests. The sixteen most sensitive and consistent tasters were retained as taste panel members. The resulting panel was diverse in age [22–42, mean: 29], sex [56% male] and nationality [7 different countries]. The panel developed a consensus vocabulary to describe beer aroma, taste and mouthfeel. Panelists were trained to identify and score 50 different attributes, using a 7-point scale to rate attributes’ intensity. The scoring sheet is included as Supplementary Data  3 . Sensory assessments took place between 10–12 a.m. The beers were served in black-colored glasses. Per session, between 5 and 12 beers of the same style were tasted at 12 °C to 16 °C. Two reference beers were added to each set and indicated as ‘Reference 1 & 2’, allowing panel members to calibrate their ratings. Not all panelists were present at every tasting. Scores were scaled by standard deviation and mean-centered per taster. Values are represented as z-scores and clustered by Euclidean distance. Pairwise Spearman correlations were calculated between taste and aroma sensory attributes. Panel consistency was evaluated by repeating samples on different sessions and performing ANOVA to identify differences, using the ‘stats’ package (v4.2.2) in R (for package information, see Supplementary Table  S8 ).

Online reviews from a public database

The ‘scrapy’ package in Python (v3.6) (for package information, see Supplementary Table  S8 ). was used to collect 232,288 online reviews (mean=922, min=6, max=5343) from RateBeer, an online beer review database. Each review entry comprised 5 numerical scores (appearance, aroma, taste, palate and overall quality) and an optional review text. The total number of reviews per reviewer was collected separately. Numerical scores were scaled and centered per rater, and mean scores were calculated per beer.

For the review texts, the language was estimated using the packages ‘langdetect’ and ‘langid’ in Python. Reviews that were classified as English by both packages were kept. Reviewers with fewer than 100 entries overall were discarded. 181,025 reviews from >6000 reviewers from >40 countries remained. Text processing was done using the ‘nltk’ package in Python. Texts were corrected for slang and misspellings; proper nouns and rare words that are relevant to the beer context were specified and kept as-is (‘Chimay’,’Lambic’, etc.). A dictionary of semantically similar sensorial terms, for example ‘floral’ and ‘flower’, was created and collapsed together into one term. Words were stemmed and lemmatized to avoid identifying words such as ‘acid’ and ‘acidity’ as separate terms. Numbers and punctuation were removed.

Sentences from up to 50 randomly chosen reviews per beer were manually categorized according to the aspect of beer they describe (appearance, aroma, taste, palate, overall quality—not to be confused with the 5 numerical scores described above) or flagged as irrelevant if they contained no useful information. If a beer contained fewer than 50 reviews, all reviews were manually classified. This labeled data set was used to train a model that classified the rest of the sentences for all beers 91 . Sentences describing taste and aroma were extracted, and term frequency–inverse document frequency (TFIDF) was implemented to calculate enrichment scores for sensorial words per beer.

The sex of the tasting subject was not considered when building our sensory database. Instead, results from different panelists were averaged, both for our trained panel (56% male, 44% female) and the RateBeer reviews (70% male, 30% female for RateBeer as a whole).

Beer price collection and processing

Beer prices were collected from the following stores: Colruyt, Delhaize, Total Wine, BeerHawk, The Belgian Beer Shop, The Belgian Shop, and Beer of Belgium. Where applicable, prices were converted to Euros and normalized per liter. Spearman correlations were calculated between these prices and mean overall appreciation scores from RateBeer and the taste panel, respectively.

Pairwise Spearman Rank correlations were calculated between all sensory properties.

Machine learning models

Predictive modeling of sensory profiles from chemical data.

Regression models were constructed to predict (a) trained panel scores for beer flavors and quality from beer chemical profiles and (b) public reviews’ appreciation scores from beer chemical profiles. Z-scores were used to represent sensory attributes in both data sets. Chemical properties with log-normal distributions (Shapiro-Wilk test, p  <  0.05 ) were log-transformed. Missing chemical measurements (0.1% of all data) were replaced with mean values per attribute. Observations from 250 beers were randomly separated into a training set (70%, 175 beers) and a test set (30%, 75 beers), stratified per beer style. Chemical measurements (p = 231) were normalized based on the training set average and standard deviation. In total, three linear regression-based models: linear regression with first-order interaction terms (LR), lasso regression with first-order interaction terms (Lasso) and partial least squares regression (PLSR); five decision tree models, Adaboost regressor (ABR), Extra Trees (ET), Gradient Boosting regressor (GBR), Random Forest (RF) and XGBoost regressor (XGBR); one support vector machine model (SVR) and one artificial neural network model (ANN) were trained. The models were implemented using the ‘scikit-learn’ package (v1.2.2) and ‘xgboost’ package (v1.7.3) in Python (v3.9.16). Models were trained, and hyperparameters optimized, using five-fold cross-validated grid search with the coefficient of determination (R 2 ) as the evaluation metric. The ANN (scikit-learn’s MLPRegressor) was optimized using Bayesian Tree-Structured Parzen Estimator optimization with the ‘Optuna’ Python package (v3.2.0). Individual models were trained per attribute, and a multi-output model was trained on all attributes simultaneously.

Model dissection

GBR was found to outperform other methods, resulting in models with the highest average R 2 values in both trained panel and public review data sets. Impurity-based rankings of the most important predictors for each predicted sensorial trait were obtained using the ‘scikit-learn’ package. To observe the relationships between these chemical properties and their predicted targets, partial dependence plots (PDP) were constructed for the six most important predictors of consumer appreciation 74 , 75 .

The ‘SHAP’ package in Python (v0.41.0) was implemented to provide an alternative ranking of predictor importance and to visualize the predictors’ effects as a function of their concentration 68 .

Validation of causal chemical properties

To validate the effects of the most important model features on predicted sensory attributes, beers were spiked with the chemical compounds identified by the models and descriptive sensory analyses were carried out according to the American Society of Brewing Chemists (ASBC) protocol 90 .

Compound spiking was done 30 min before tasting. Compounds were spiked into fresh beer bottles, that were immediately resealed and inverted three times. Fresh bottles of beer were opened for the same duration, resealed, and inverted thrice, to serve as controls. Pairs of spiked samples and controls were served simultaneously, chilled and in dark glasses as outlined in the Trained panel section above. Tasters were instructed to select the glass with the higher flavor intensity for each attribute (directional difference test 92 ) and to select the glass they prefer.

The final concentration after spiking was equal to the within-style average, after normalizing by ethanol concentration. This was done to ensure balanced flavor profiles in the final spiked beer. The same methods were applied to improve a non-alcoholic beer. Compounds were the following: ethyl acetate (Merck KGaA, W241415), ethyl hexanoate (Merck KGaA, W243906), isoamyl acetate (Merck KGaA, W205508), phenethyl acetate (Merck KGaA, W285706), ethanol (96%, Colruyt), glycerol (Merck KGaA, W252506), lactic acid (Merck KGaA, 261106).

Significant differences in preference or perceived intensity were determined by performing the two-sided binomial test on each attribute.

Reporting summary

Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Data availability

The data that support the findings of this work are available in the Supplementary Data files and have been deposited to Zenodo under accession code 10653704 93 . The RateBeer scores data are under restricted access, they are not publicly available as they are property of RateBeer (ZX Ventures, USA). Access can be obtained from the authors upon reasonable request and with permission of RateBeer (ZX Ventures, USA).  Source data are provided with this paper.

Code availability

The code for training the machine learning models, analyzing the models, and generating the figures has been deposited to Zenodo under accession code 10653704 93 .

Tieman, D. et al. A chemical genetic roadmap to improved tomato flavor. Science 355 , 391–394 (2017).

Article   ADS   CAS   PubMed   Google Scholar  

Plutowska, B. & Wardencki, W. Application of gas chromatography–olfactometry (GC–O) in analysis and quality assessment of alcoholic beverages – A review. Food Chem. 107 , 449–463 (2008).

Article   CAS   Google Scholar  

Legin, A., Rudnitskaya, A., Seleznev, B. & Vlasov, Y. Electronic tongue for quality assessment of ethanol, vodka and eau-de-vie. Anal. Chim. Acta 534 , 129–135 (2005).

Loutfi, A., Coradeschi, S., Mani, G. K., Shankar, P. & Rayappan, J. B. B. Electronic noses for food quality: A review. J. Food Eng. 144 , 103–111 (2015).

Ahn, Y.-Y., Ahnert, S. E., Bagrow, J. P. & Barabási, A.-L. Flavor network and the principles of food pairing. Sci. Rep. 1 , 196 (2011).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bartoshuk, L. M. & Klee, H. J. Better fruits and vegetables through sensory analysis. Curr. Biol. 23 , R374–R378 (2013).

Article   CAS   PubMed   Google Scholar  

Piggott, J. R. Design questions in sensory and consumer science. Food Qual. Prefer. 3293 , 217–220 (1995).

Article   Google Scholar  

Kermit, M. & Lengard, V. Assessing the performance of a sensory panel-panellist monitoring and tracking. J. Chemom. 19 , 154–161 (2005).

Cook, D. J., Hollowood, T. A., Linforth, R. S. T. & Taylor, A. J. Correlating instrumental measurements of texture and flavour release with human perception. Int. J. Food Sci. Technol. 40 , 631–641 (2005).

Chinchanachokchai, S., Thontirawong, P. & Chinchanachokchai, P. A tale of two recommender systems: The moderating role of consumer expertise on artificial intelligence based product recommendations. J. Retail. Consum. Serv. 61 , 1–12 (2021).

Ross, C. F. Sensory science at the human-machine interface. Trends Food Sci. Technol. 20 , 63–72 (2009).

Chambers, E. IV & Koppel, K. Associations of volatile compounds with sensory aroma and flavor: The complex nature of flavor. Molecules 18 , 4887–4905 (2013).

Pinu, F. R. Metabolomics—The new frontier in food safety and quality research. Food Res. Int. 72 , 80–81 (2015).

Danezis, G. P., Tsagkaris, A. S., Brusic, V. & Georgiou, C. A. Food authentication: state of the art and prospects. Curr. Opin. Food Sci. 10 , 22–31 (2016).

Shepherd, G. M. Smell images and the flavour system in the human brain. Nature 444 , 316–321 (2006).

Meilgaard, M. C. Prediction of flavor differences between beers from their chemical composition. J. Agric. Food Chem. 30 , 1009–1017 (1982).

Xu, L. et al. Widespread receptor-driven modulation in peripheral olfactory coding. Science 368 , eaaz5390 (2020).

Kupferschmidt, K. Following the flavor. Science 340 , 808–809 (2013).

Billesbølle, C. B. et al. Structural basis of odorant recognition by a human odorant receptor. Nature 615 , 742–749 (2023).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Smith, B. Perspective: Complexities of flavour. Nature 486 , S6–S6 (2012).

Pfister, P. et al. Odorant receptor inhibition is fundamental to odor encoding. Curr. Biol. 30 , 2574–2587 (2020).

Moskowitz, H. W., Kumaraiah, V., Sharma, K. N., Jacobs, H. L. & Sharma, S. D. Cross-cultural differences in simple taste preferences. Science 190 , 1217–1218 (1975).

Eriksson, N. et al. A genetic variant near olfactory receptor genes influences cilantro preference. Flavour 1 , 22 (2012).

Ferdenzi, C. et al. Variability of affective responses to odors: Culture, gender, and olfactory knowledge. Chem. Senses 38 , 175–186 (2013).

Article   PubMed   Google Scholar  

Lawless, H. T. & Heymann, H. Sensory evaluation of food: Principles and practices. (Springer, New York, NY). https://doi.org/10.1007/978-1-4419-6488-5 (2010).

Colantonio, V. et al. Metabolomic selection for enhanced fruit flavor. Proc. Natl. Acad. Sci. 119 , e2115865119 (2022).

Fritz, F., Preissner, R. & Banerjee, P. VirtualTaste: a web server for the prediction of organoleptic properties of chemical compounds. Nucleic Acids Res 49 , W679–W684 (2021).

Tuwani, R., Wadhwa, S. & Bagler, G. BitterSweet: Building machine learning models for predicting the bitter and sweet taste of small molecules. Sci. Rep. 9 , 1–13 (2019).

Dagan-Wiener, A. et al. Bitter or not? BitterPredict, a tool for predicting taste from chemical structure. Sci. Rep. 7 , 1–13 (2017).

Pallante, L. et al. Toward a general and interpretable umami taste predictor using a multi-objective machine learning approach. Sci. Rep. 12 , 1–11 (2022).

Malavolta, M. et al. A survey on computational taste predictors. Eur. Food Res. Technol. 248 , 2215–2235 (2022).

Lee, B. K. et al. A principal odor map unifies diverse tasks in olfactory perception. Science 381 , 999–1006 (2023).

Mayhew, E. J. et al. Transport features predict if a molecule is odorous. Proc. Natl. Acad. Sci. 119 , e2116576119 (2022).

Niu, Y. et al. Sensory evaluation of the synergism among ester odorants in light aroma-type liquor by odor threshold, aroma intensity and flash GC electronic nose. Food Res. Int. 113 , 102–114 (2018).

Yu, P., Low, M. Y. & Zhou, W. Design of experiments and regression modelling in food flavour and sensory analysis: A review. Trends Food Sci. Technol. 71 , 202–215 (2018).

Oladokun, O. et al. The impact of hop bitter acid and polyphenol profiles on the perceived bitterness of beer. Food Chem. 205 , 212–220 (2016).

Linforth, R., Cabannes, M., Hewson, L., Yang, N. & Taylor, A. Effect of fat content on flavor delivery during consumption: An in vivo model. J. Agric. Food Chem. 58 , 6905–6911 (2010).

Guo, S., Na Jom, K. & Ge, Y. Influence of roasting condition on flavor profile of sunflower seeds: A flavoromics approach. Sci. Rep. 9 , 11295 (2019).

Ren, Q. et al. The changes of microbial community and flavor compound in the fermentation process of Chinese rice wine using Fagopyrum tataricum grain as feedstock. Sci. Rep. 9 , 3365 (2019).

Hastie, T., Friedman, J. & Tibshirani, R. The Elements of Statistical Learning. (Springer, New York, NY). https://doi.org/10.1007/978-0-387-21606-5 (2001).

Dietz, C., Cook, D., Huismann, M., Wilson, C. & Ford, R. The multisensory perception of hop essential oil: a review. J. Inst. Brew. 126 , 320–342 (2020).

CAS   Google Scholar  

Roncoroni, Miguel & Verstrepen, Kevin Joan. Belgian Beer: Tested and Tasted. (Lannoo, 2018).

Meilgaard, M. Flavor chemistry of beer: Part II: Flavor and threshold of 239 aroma volatiles. in (1975).

Bokulich, N. A. & Bamforth, C. W. The microbiology of malting and brewing. Microbiol. Mol. Biol. Rev. MMBR 77 , 157–172 (2013).

Dzialo, M. C., Park, R., Steensels, J., Lievens, B. & Verstrepen, K. J. Physiology, ecology and industrial applications of aroma formation in yeast. FEMS Microbiol. Rev. 41 , S95–S128 (2017).

Article   PubMed   PubMed Central   Google Scholar  

Datta, A. et al. Computer-aided food engineering. Nat. Food 3 , 894–904 (2022).

American Society of Brewing Chemists. Beer Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A.).

Olaniran, A. O., Hiralal, L., Mokoena, M. P. & Pillay, B. Flavour-active volatile compounds in beer: production, regulation and control. J. Inst. Brew. 123 , 13–23 (2017).

Verstrepen, K. J. et al. Flavor-active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Meilgaard, M. C. Flavour chemistry of beer. part I: flavour interaction between principal volatiles. Master Brew. Assoc. Am. Tech. Q 12 , 107–117 (1975).

Briggs, D. E., Boulton, C. A., Brookes, P. A. & Stevens, R. Brewing 227–254. (Woodhead Publishing). https://doi.org/10.1533/9781855739062.227 (2004).

Bossaert, S., Crauwels, S., De Rouck, G. & Lievens, B. The power of sour - A review: Old traditions, new opportunities. BrewingScience 72 , 78–88 (2019).

Google Scholar  

Verstrepen, K. J. et al. Flavor active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Snauwaert, I. et al. Microbial diversity and metabolite composition of Belgian red-brown acidic ales. Int. J. Food Microbiol. 221 , 1–11 (2016).

Spitaels, F. et al. The microbial diversity of traditional spontaneously fermented lambic beer. PLoS ONE 9 , e95384 (2014).

Blanco, C. A., Andrés-Iglesias, C. & Montero, O. Low-alcohol Beers: Flavor Compounds, Defects, and Improvement Strategies. Crit. Rev. Food Sci. Nutr. 56 , 1379–1388 (2016).

Jackowski, M. & Trusek, A. Non-Alcohol. beer Prod. – Overv. 20 , 32–38 (2018).

Takoi, K. et al. The contribution of geraniol metabolism to the citrus flavour of beer: Synergy of geraniol and β-citronellol under coexistence with excess linalool. J. Inst. Brew. 116 , 251–260 (2010).

Kroeze, J. H. & Bartoshuk, L. M. Bitterness suppression as revealed by split-tongue taste stimulation in humans. Physiol. Behav. 35 , 779–783 (1985).

Mennella, J. A. et al. A spoonful of sugar helps the medicine go down”: Bitter masking bysucrose among children and adults. Chem. Senses 40 , 17–25 (2015).

Wietstock, P., Kunz, T., Perreira, F. & Methner, F.-J. Metal chelation behavior of hop acids in buffered model systems. BrewingScience 69 , 56–63 (2016).

Sancho, D., Blanco, C. A., Caballero, I. & Pascual, A. Free iron in pale, dark and alcohol-free commercial lager beers. J. Sci. Food Agric. 91 , 1142–1147 (2011).

Rodrigues, H. & Parr, W. V. Contribution of cross-cultural studies to understanding wine appreciation: A review. Food Res. Int. 115 , 251–258 (2019).

Korneva, E. & Blockeel, H. Towards better evaluation of multi-target regression models. in ECML PKDD 2020 Workshops (eds. Koprinska, I. et al.) 353–362 (Springer International Publishing, Cham, 2020). https://doi.org/10.1007/978-3-030-65965-3_23 .

Gastón Ares. Mathematical and Statistical Methods in Food Science and Technology. (Wiley, 2013).

Grinsztajn, L., Oyallon, E. & Varoquaux, G. Why do tree-based models still outperform deep learning on tabular data? Preprint at http://arxiv.org/abs/2207.08815 (2022).

Gries, S. T. Statistics for Linguistics with R: A Practical Introduction. in Statistics for Linguistics with R (De Gruyter Mouton, 2021). https://doi.org/10.1515/9783110718256 .

Lundberg, S. M. et al. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2 , 56–67 (2020).

Ickes, C. M. & Cadwallader, K. R. Effects of ethanol on flavor perception in alcoholic beverages. Chemosens. Percept. 10 , 119–134 (2017).

Kato, M. et al. Influence of high molecular weight polypeptides on the mouthfeel of commercial beer. J. Inst. Brew. 127 , 27–40 (2021).

Wauters, R. et al. Novel Saccharomyces cerevisiae variants slow down the accumulation of staling aldehydes and improve beer shelf-life. Food Chem. 398 , 1–11 (2023).

Li, H., Jia, S. & Zhang, W. Rapid determination of low-level sulfur compounds in beer by headspace gas chromatography with a pulsed flame photometric detector. J. Am. Soc. Brew. Chem. 66 , 188–191 (2008).

Dercksen, A., Laurens, J., Torline, P., Axcell, B. C. & Rohwer, E. Quantitative analysis of volatile sulfur compounds in beer using a membrane extraction interface. J. Am. Soc. Brew. Chem. 54 , 228–233 (1996).

Molnar, C. Interpretable Machine Learning: A Guide for Making Black-Box Models Interpretable. (2020).

Zhao, Q. & Hastie, T. Causal interpretations of black-box models. J. Bus. Econ. Stat. Publ. Am. Stat. Assoc. 39 , 272–281 (2019).

Article   MathSciNet   Google Scholar  

Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning. (Springer, 2019).

Labrado, D. et al. Identification by NMR of key compounds present in beer distillates and residual phases after dealcoholization by vacuum distillation. J. Sci. Food Agric. 100 , 3971–3978 (2020).

Lusk, L. T., Kay, S. B., Porubcan, A. & Ryder, D. S. Key olfactory cues for beer oxidation. J. Am. Soc. Brew. Chem. 70 , 257–261 (2012).

Gonzalez Viejo, C., Torrico, D. D., Dunshea, F. R. & Fuentes, S. Development of artificial neural network models to assess beer acceptability based on sensory properties using a robotic pourer: A comparative model approach to achieve an artificial intelligence system. Beverages 5 , 33 (2019).

Gonzalez Viejo, C., Fuentes, S., Torrico, D. D., Godbole, A. & Dunshea, F. R. Chemical characterization of aromas in beer and their effect on consumers liking. Food Chem. 293 , 479–485 (2019).

Gilbert, J. L. et al. Identifying breeding priorities for blueberry flavor using biochemical, sensory, and genotype by environment analyses. PLOS ONE 10 , 1–21 (2015).

Goulet, C. et al. Role of an esterase in flavor volatile variation within the tomato clade. Proc. Natl. Acad. Sci. 109 , 19009–19014 (2012).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Borisov, V. et al. Deep Neural Networks and Tabular Data: A Survey. IEEE Trans. Neural Netw. Learn. Syst. 1–21 https://doi.org/10.1109/TNNLS.2022.3229161 (2022).

Statista. Statista Consumer Market Outlook: Beer - Worldwide.

Seitz, H. K. & Stickel, F. Molecular mechanisms of alcoholmediated carcinogenesis. Nat. Rev. Cancer 7 , 599–612 (2007).

Voordeckers, K. et al. Ethanol exposure increases mutation rate through error-prone polymerases. Nat. Commun. 11 , 3664 (2020).

Goelen, T. et al. Bacterial phylogeny predicts volatile organic compound composition and olfactory response of an aphid parasitoid. Oikos 129 , 1415–1428 (2020).

Article   ADS   Google Scholar  

Reher, T. et al. Evaluation of hop (Humulus lupulus) as a repellent for the management of Drosophila suzukii. Crop Prot. 124 , 104839 (2019).

Stein, S. E. An integrated method for spectrum extraction and compound identification from gas chromatography/mass spectrometry data. J. Am. Soc. Mass Spectrom. 10 , 770–781 (1999).

American Society of Brewing Chemists. Sensory Analysis Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A., 1992).

McAuley, J., Leskovec, J. & Jurafsky, D. Learning Attitudes and Attributes from Multi-Aspect Reviews. Preprint at https://doi.org/10.48550/arXiv.1210.3926 (2012).

Meilgaard, M. C., Carr, B. T. & Carr, B. T. Sensory Evaluation Techniques. (CRC Press, Boca Raton). https://doi.org/10.1201/b16452 (2014).

Schreurs, M. et al. Data from: Predicting and improving complex beer flavor through machine learning. Zenodo https://doi.org/10.5281/zenodo.10653704 (2024).

Download references

Acknowledgements

We thank all lab members for their discussions and thank all tasting panel members for their contributions. Special thanks go out to Dr. Karin Voordeckers for her tremendous help in proofreading and improving the manuscript. M.S. was supported by a Baillet-Latour fellowship, L.C. acknowledges financial support from KU Leuven (C16/17/006), F.A.T. was supported by a PhD fellowship from FWO (1S08821N). Research in the lab of K.J.V. is supported by KU Leuven, FWO, VIB, VLAIO and the Brewing Science Serves Health Fund. Research in the lab of T.W. is supported by FWO (G.0A51.15) and KU Leuven (C16/17/006).

Author information

These authors contributed equally: Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni.

Authors and Affiliations

VIB—KU Leuven Center for Microbiology, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni, Lloyd Cool, Beatriz Herrera-Malaver, Florian A. Theßeling & Kevin J. Verstrepen

CMPG Laboratory of Genetics and Genomics, KU Leuven, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Leuven Institute for Beer Research (LIBR), Gaston Geenslaan 1, B-3001, Leuven, Belgium

Laboratory of Socioecology and Social Evolution, KU Leuven, Naamsestraat 59, B-3000, Leuven, Belgium

Lloyd Cool, Christophe Vanderaa & Tom Wenseleers

VIB Bioinformatics Core, VIB, Rijvisschestraat 120, B-9052, Ghent, Belgium

Łukasz Kreft & Alexander Botzki

AB InBev SA/NV, Brouwerijplein 1, B-3000, Leuven, Belgium

Philippe Malcorps & Luk Daenen

You can also search for this author in PubMed   Google Scholar

Contributions

S.P., M.S. and K.J.V. conceived the experiments. S.P., M.S. and K.J.V. designed the experiments. S.P., M.S., M.R., B.H. and F.A.T. performed the experiments. S.P., M.S., L.C., C.V., L.K., A.B., P.M., L.D., T.W. and K.J.V. contributed analysis ideas. S.P., M.S., L.C., C.V., T.W. and K.J.V. analyzed the data. All authors contributed to writing the manuscript.

Corresponding author

Correspondence to Kevin J. Verstrepen .

Ethics declarations

Competing interests.

K.J.V. is affiliated with bar.on. The other authors declare no competing interests.

Peer review

Peer review information.

Nature Communications thanks Florian Bauer, Andrew John Macintosh and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, peer review file, description of additional supplementary files, supplementary data 1, supplementary data 2, supplementary data 3, supplementary data 4, supplementary data 5, supplementary data 6, supplementary data 7, reporting summary, source data, source data, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Schreurs, M., Piampongsant, S., Roncoroni, M. et al. Predicting and improving complex beer flavor through machine learning. Nat Commun 15 , 2368 (2024). https://doi.org/10.1038/s41467-024-46346-0

Download citation

Received : 30 October 2023

Accepted : 21 February 2024

Published : 26 March 2024

DOI : https://doi.org/10.1038/s41467-024-46346-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Scribbr Citation Generator

Accurate APA, MLA, Chicago, and Harvard citations, verified by experts, trusted by millions

Scribbr for Chrome: Your shortcut to citations

Cite any page or article with a single click right from your browser. The extension does the hard work for you by automatically grabbing the title, author(s), publication date, and everything else needed to whip up the perfect citation.

Perfectly formatted references every time

Inaccurate citations can cost you points on your assignments, so our seasoned citation experts have invested countless hours in perfecting Scribbr’s citation generator algorithms. We’re proud to be recommended by teachers and universities worldwide.

Enjoy a citation generator without flashy ads

Staying focused is already difficult enough, so unlike other citation generators, Scribbr won’t slow you down with flashing banner ads and video pop-ups. That’s a promise!

Citation Generator features you'll love

Look up your source by its title, URL, ISBN, or DOI, and let Scribbr find and fill in all the relevant information automatically.

APA, MLA, Chicago, and Harvard

Generate flawless citations according to the official APA, MLA, Chicago, Harvard style, or many other rules.

Export to Word

When your reference list is complete, export it to Word. We’ll apply the official formatting guidelines automatically.

Lists and folders

Create separate reference lists for each of your assignments to stay organized. You can also group related lists into folders.

Export to Bib(La)TeX

Are you using a LaTex editor like Overleaf? If so, you can easily export your references in Bib(La)TeX format with a single click.

Custom fonts

Change the typeface used for your reference list to match the rest of your document. Options include Times New Roman, Arial, and Calibri.

Industry-standard technology

Scribbr’s Citation Generator is built using the same citation software (CSL) as Mendeley and Zotero, but with an added layer for improved accuracy.

Annotations

Describe or evaluate your sources in annotations, and Scribbr will generate a perfectly formatted annotated bibliography .

Citation guides

Scribbr’s popular guides and videos will help you understand everything related to finding, evaluating, and citing sources.

Secure backup

Your work is saved automatically after every change and stored securely in your Scribbr account.

• Introduction
• Finding sources

Evaluating sources

• Integrating sources

Citing sources

Tools and resources, a quick guide to working with sources.

Working with sources is an important skill that you’ll need throughout your academic career.

It includes knowing how to find relevant sources, assessing their authority and credibility, and understanding how to integrate sources into your work with proper referencing.

This quick guide will help you get started!

Finding relevant sources

Sources commonly used in academic writing include academic journals, scholarly books, websites, newspapers, and encyclopedias. There are three main places to look for such sources:

• Research databases: Databases can be general or subject-specific. To get started, check out this list of databases by academic discipline . Another good starting point is Google Scholar .
• Your institution’s library: Use your library’s database to narrow down your search using keywords to find relevant articles, books, and newspapers matching your topic.
• Other online resources: Consult popular online sources like websites, blogs, or Wikipedia to find background information. Be sure to carefully evaluate the credibility of those online sources.

When using academic databases or search engines, you can use Boolean operators to refine your results.

Generate APA, MLA, Chicago, and Harvard citations in seconds

Get started

In academic writing, your sources should be credible, up to date, and relevant to your research topic. Useful approaches to evaluating sources include the CRAAP test and lateral reading.

CRAAP is an abbreviation that reminds you of a set of questions to ask yourself when evaluating information.

• Currency: Does the source reflect recent research?
• Relevance: Is the source related to your research topic?
• Authority: Is it a respected publication? Is the author an expert in their field?
• Accuracy: Does the source support its arguments and conclusions with evidence?
• Purpose: What is the author’s intention?

Lateral reading

Lateral reading means comparing your source to other sources. This allows you to:

• Verify evidence
• Contextualize information
• Find potential weaknesses

If a source is using methods or drawing conclusions that are incompatible with other research in its field, it may not be reliable.

Integrating sources into your work

Once you have found information that you want to include in your paper, signal phrases can help you to introduce it. Here are a few examples:

Following the signal phrase, you can choose to quote, paraphrase or summarize the source.

• Quoting : This means including the exact words of another source in your paper. The quoted text must be enclosed in quotation marks or (for longer quotes) presented as a block quote . Quote a source when the meaning is difficult to convey in different words or when you want to analyze the language itself.
• Paraphrasing : This means putting another person’s ideas into your own words. It allows you to integrate sources more smoothly into your text, maintaining a consistent voice. It also shows that you have understood the meaning of the source.
• Summarizing : This means giving an overview of the essential points of a source. Summaries should be much shorter than the original text. You should describe the key points in your own words and not quote from the original text.

Whenever you quote, paraphrase, or summarize a source, you must include a citation crediting the original author.

Citing your sources is important because it:

• Allows you to avoid plagiarism
• Establishes the credentials of your sources
• Backs up your arguments with evidence
• Allows your reader to verify the legitimacy of your conclusions

The most common citation styles are APA, MLA, and Chicago style. Each citation style has specific rules for formatting citations.

Generate APA, MLA, Chicago,  and Harvard citations in seconds

Scribbr offers tons of tools and resources to make working with sources easier and faster. Take a look at our top picks:

• Citation Generator: Automatically generate accurate references and in-text citations using Scribbr’s APA Citation Generator, MLA Citation Generator , Harvard Referencing Generator , and Chicago Citation Generator .
• Plagiarism Checker : Detect plagiarism in your paper using the most accurate Turnitin-powered plagiarism software available to students.
• AI Proofreader: Upload and improve unlimited documents and earn higher grades on your assignments. Try it for free!
• Paraphrasing tool: Avoid accidental plagiarism and make your text sound better.
• Grammar checker : Eliminate pesky spelling and grammar mistakes.
• Summarizer: Read more in less time. Distill lengthy and complex texts down to their key points.
• AI detector: Find out if your text was written with ChatGPT or any other AI writing tool. ChatGPT 2 & ChatGPT 3 supported.
• Proofreading services : Have a human editor improve your writing.
• Citation checker: Check your work for citation errors and missing citations.
• Knowledge Base : Explore hundreds of articles, bite-sized videos, time-saving templates, and handy checklists that guide you through the process of research, writing, and citation.