This is the most memorable abstract I've ever read.

Smith, Gordon C. S., and Jill P. Pell. 2003. “Parachute Use to Prevent Death and Major Trauma Related to Gravitational Challenge: Systematic Review of Randomised Controlled Trials.” BMJ 327 (7429): 1459–61. https://doi.org/10.1136/bmj.327.7429.1459.

Objectives: To determine whether parachutes are effective in preventing major trauma related to gravitational challenge.

Design: Systematic review of randomised controlled trials.

Data sources: Medline, Web of Science, Embase, and the Cochrane Library databases; appropriate internet sites and citation lists.

Study selection: Studies showing the effects of using a parachute during free fall.

Main outcome measure: Death or major trauma, defined as an injury severity score > 15.

Results: We were unable to identify any randomised controlled trials of parachute intervention.

Conclusions: As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.

I'm well into the thousands of research papers read. Understood a fair fraction of them. Couldn't tell you what the abstract was for any. The memorable parts are the ideas and the results within the paper.

Some titles stick. "Attention is all you need". "SSA is functional programming". Plus more people see the title than read the abstract. Only one I got to name myself was "Shared Memory Remote Procedure Calls" which I now realise is in exactly the same theme of given the title, you don't need the abstract or the contents.

An abstract and an introduction are two different things. For me, as a reader, the main role of the abstract is to help me to decide whether to even look at the rest of the paper, let alone read it. I want to know what is in the paper that I will not find in other papers that are competing for my time. Everything should be very clear: the context, the main research contributions and/or findings, novel aspects of the approach, limitations; plus any other information that in the author's view should motivate me to read the paper. Then if I do read the paper, there should not be any surprises: the content that the abstract promised should be present and, ideally, all of the valuable content in the paper should have been advertised in the abstract.

Different fields use different standard forms or styles for abstracts. If you have not already done so you can get a feel for this by looking at a random sample of the top 100 most-cited papers in your field (or your target conferences, journals, arxiv categories) and quickly compare to top 100 in a couple of disparate fields (e.g. biochemistry, economics, anthropology, mathematics).

Of course, there are good reasons to read a paper besides the quality and content of the abstract. Consequently, there are well known papers with non-standard abstracts. I recommend mastering the basics first.

Abstracts are not meant to be entertaining or memorable; they're meant to be highly functional. When you're reading a billion papers, a good abstract is important because it lets you get the gist of the paper and whether or not it's relevant to what you're researching in about 10 seconds. Make your abstract extremely clear and well structured about what you did, why, and your biggest finding, and the worry about giving it a good hook.

Hard to separate the memorability of the abstract from the underlying material.

IMO the most memorable papers contain some unexpected simplification: a complicated problem turns out to reduce to something much simpler [0], perhaps for a counterintuitive reason [1].

So a memorable abstract should advertise that the paper contains some cute little trick. This may not help you actually publish though

[0]: Attack the RLHF problem with a simple classification loss: https://arxiv.org/abs/2305.18290

[1]: Frustratingly Easy Meta-Embeddings: https://arxiv.org/abs/1804.05262