How scholars trade off paper attributes inside a recommendation
At-a-glance summary
What we studied: How scholars trade off venue, recency, originality, and author familiarity when choosing which paper to read—within a set of papers already on their current research topic.
What we found:
Scholars don’t look at one attribute at a time; they weigh venue, recency, originality, and author familiarity together when choosing between otherwise on-topic papers.
Specialized journals in their niche were preferred over broad or cross-disciplinary flagships.
Newer papers (published within roughly the past year) were more attractive than very old work—but it’s not necessary to be hot-off-the-press.
Originality mattered at least as much as author fame. Scholars rewarded new ideas and creative extensions more strongly than simply recognizing a famous name.
Why this mattered: Previous work had shown that topical relevance is the first gate for recommendations. But once papers clear that bar, it wasn’t clear which other attributes actually sway scholars’ choices. This study quantified those trade-offs so we could make more principled ranking algorithm decisions.
Impact: The results informed attribute definition and weighting in the recommendation model.
Project background
Prior qualitative research showed that when scholars scan recommended papers, they first ask “Is this on my current project?” and only then look at other signals such as date, venue, and author. Once a paper is clearly on-topic, however, the decision becomes more nuanced: scholars often have to choose which of several relevant papers to read next. To understand this decision space, we ran a choice-based conjoint survey to estimate how scholars trade off these attributes when deciding what to read next.
Research questions
Which attributes most strongly drive choice: venue, recency, originality, or author familiarity?
How important is each of these attributes relative to the others?
What patterns in these trade-offs could inform better recommendation and discovery experiences?
Methods & participants
We recruited 278 scholars from an academic literature platform:
faculty, postdocs, and graduate students
across humanities, social sciences, and STEM.
Participants completed an online choice-based conjoint survey. Each task showed a short description of two hypothetical papers and asked:
Each paper varied along four attributes, with levels like:
Publication date – from very recent (months) to several years old.
Journal type – unfamiliar journal, specialized niche journal, flagship field journal, or prestigious cross-disciplinary journal.
Originality – uses established approaches; extends existing ideas in a creative way; introduces a completely new idea/method/finding.
Author familiarity – highly prominent author; recognizable but not highly prominent; no authors recognized.
By analyzing repeated choices across tasks, we estimated:
attribute importance (how much each dimension influenced choice)
part-worth utilities (how attractive each level was within an attribute)
Findings
Finding 1. Venue fit matters more than venue prestige
Within journal type:
Specialized journals covering a niche area were most preferred.
Broad flagship field journals and prestigious cross-disciplinary venues were attractive, but not as attractive as niche venues that tightly matched the scholar’s area.
Unfamiliar journals were least preferred.
Interpretation: Scholars are not simply chasing prestige. When papers are already on-topic, they reward venues that are both reputable and highly aligned to their specific subfield. Venue appears to function as fit × quality, not prestige alone.
Finding 2. Scholars prefer “fresh” over “old”—but not necessarily “brand-new”
Publication date was the second strongest driver of choice, on par with journal type and originality.
Within recency:
Papers published within the past year were most attractive.
Work from one to two years ago was still appealing, but less so.
Very old papers (several years back) dropped sharply in attractiveness.
The very newest papers were not clearly preferred over “fresh within a year” papers
Because every choice task fixed topical relevance (“current specific research focus”), we interpret this as:
scholars want up-to-date work
that has had a little time to circulate, not necessarily day-of-release
This suggests a recency window where work feels both relevant and stable enough to be worth attention.
Finding 3. Originality is a first-class signal, not a “nice-to-have”
Originality mattered as much as recency and more than author familiarity.
Within originality:
Completely new ideas, methods, or findings were most preferred
Creative extensions of existing ideas were close behind
Routine applications of known approaches were least attractive
Scholars are not just looking for “more of the same”: once a paper is on-topic, many actively seek work that moves the conversation forward. This also highlights that novelty is an important cognitive signal—even though most recommendation systems today do not explicitly represent it.
Finding 4. Author familiarity matters—but less than venue, recency, and originality
Author familiarity had the smallest influence, though still meaningful:
Prominent authors increased likelihood of selection
Recognizable-but-not-famous authors performed similarly
Unknown authors were least attractive
Author reputation can nudge a decision, but it does not outweigh venue alignment, recency, or originality.
Implications & impact
The findings from the study suggest promising modeling directions:
Weight venue fit strongly. Specialized, reputable journals in a scholar’s niche appear particularly valuable, beyond prestige alone.
Apply a shaped recency curve. Favor work in a “fresh but not ancient” window instead of only prioritizing the newest possible item.
Elevate novelty signals. Where feasible, approximate originality (e.g., new methods, new framing, unusual citation recombinations).
Keep author fame as a secondary factor. Helpful within close decisions, but not a dominant factor.
Conclusion
This conjoint study zoomed in on a specific moment in scholarly discovery: choosing between two papers already relevant to one’s current project. In that moment, scholars systematically trade off venue fit, recency, originality, and author familiarity—with niche venues, fresh (but not ultra-new) publication dates, and genuine novelty often outweighing pure prestige or fame.
For recommender and discovery systems, the takeaway is clear: once topical relevance is handled, venue, time, and contribution quality meaningfully shape attention. Aligning ranking with these trade-offs can better match real scholarly judgment and help researchers spend limited attention where it matters most.