AI--Friend or Foe?

Wow, there is a lot going on with applications of Artificial Intelligence to text. Will it make human writers obsolete? Will  students use it to cheat? Can it help scientists understand the onslaught of papers we see everyday?

The answers to those questions, in my opinion at least, are no, yes, and yes. And to help answer the last question, I took a look at Explainpaper, a website that allows you to upload a paper, highlight the confusing text, and get a simplified explanation. The website was built by co-developers Aman Jha and Jade using GPT-3, a computer program family (Generative Pre-trained Transformer 3) that is getting a lot of press for another application, CHATGPT, which I will talk about in a different blog.

The first paper I uploaded was a paper by Subramanian, et al., published in PNAS October 25, 2005. This paper is entitled "Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles". The paper is well-written, and I think I could interpret it without any help, but was curious what Explainpaper would do.  

The website does not automatically interpret the whole paper. It displays the manuscript as a PDF and invites the user to highlight a section of interest for interpretation. I tried to feed the whole abstract into it however it just pinwheeled for a while so I think that might have been too much, either that or I needed a faster internet connection. I then tried the first sentence of the abstract: "Although genomewide RNA expression analysis has become a routine tool in biomedical research, extracting biological insight from such information remains a major challenge".  The interpretation by Explainpaper was: "DNA microarray technology has become a common technique used in biomedical research. However, it is difficult to take away useful information or 'biological insight' from the data generated by the microarray. It is a challenge to interpret the results and use them to gain understanding of the mechanisms of the experiment."  That seemed like a fair interpretation--however the first sentence of an abstract is usually fairly straight-forward so it was not a very rigorous test.

I then tried the first two sentences of the abstract and found out that the interpretation of the first sentence was influenced by the presence of the second sentence--perhaps there is a limit to how much it can return and so it shortened the interpretation of the first sentence in order to give an interpretation of the second sentence. It was still accurate, just shorter.

I then highlighted a part of the paper that discussed the statistical method for calculating the enrichment score in the GSEA (Genome set enrichment analysis). This section was 'denser" with reference to 'set S', and a 'running-sum statistic'. Explainpaper did a decent job of simplifying--it did seem to miss a nuance as the original paper discussed the "degree to which a set S is overrepresented at the extremes" while the translation talked about the set S being represented at the top and bottom of a list. Further down the paper, an even denser section about familywise-error rate gave some interpretation, but also gave back some sections that were nearly word for word to the original paper.

I then tried a different paper, entitled "Tissue-specific 5-hydroxymethylcytosine landscape of the human genome", by He, et al., published in Nature Communications (2021). Once again I tried the first two sentences of the abstract, which the website handled well.  Then I went into the body of the paper and found a complex section and fed that into the process.  As before, Explainpaper did a decent job of both explaining any abbreviations in the text, as well as explaining what the text actually meant. 

A small sample size, to be sure, but overall I was impressed--I think Explainpaper could be quite helpful.  The one thing I did find problematic was a very human mistake. I set up an account, and then forgot my password.  And oddly enough, I was only able to find the "reset password by sending an email" after realizing that even though there wasn't any scroll bar on the right side of the window, the reset link was off the bottom of the window. AND when the email did come, it did not take me to a reset window, it just dumped me into my account, which was great because at least I could use it, but there isn't anyway to change the password inside the account either!  So, a few growing pains perhaps, but overall I am quite intrigued by the site and look forward to trying it out again.  

Image Credit: Wellcome