What it does

A typical workflow is to ask which databases are available, explore the relevant schemas, and write a SQL query from that context. Once the results look right, they can be exported for further analysis outside the conversation. I illustrate this below with an open access analysis of publications from authors affiliated with German institutions, and from the University of Göttingen.

The full Claude chat transcript can be found here https://claude.ai/share/ec48f509-78a1-48c9-9f98-98b1c60147ee.

Costs

The BigQuery side of this entire use-case session, five queries plus the export, about 46 GB scanned, cost under $0.30, and would typically fall within Google BigQuery’s free tier of 1 TB per month. Claude usage comes on top. Run against the API with Sonnet, this session would cost roughly another $0.30–0.50; in Claude Desktop, as here, it is covered by the subscription.

Installation

Full instructions are in the GitHub repo README.

The server runs in a Docker container connected to Claude Desktop via its MCP config file. Authentication uses Google’s Application Default Credentials, so local gcloud credentials are used directly and no service account keys are needed. A Google Cloud account includes 1 TB of free queries per month.

The tool is implemented in R using {mcptools} and {ellmer}, assisted with Claude Code. It mainly uses {bigrquery} to interact with Google BigQuery, which I use in my everyday work.

Note that I only tested orion-mcp using Claude and Claude Desktop. If you find any bugs or have questions, please feel free to file an issue.

Responsible use

LLMs make mistakes. Always verify that queries return the results you intended before using them in any analysis. If you plan to use this in a publication, check the outlet’s policy on AI-assisted work and document your process accordingly.

Data and Method

This analysis reproduces and adapts the data analytics workflow underlying HOAD, as describe in Jahn (2025b). To this end, data retrieval started with obtaining journal and institutional information from preserved transformative agreement data from the cOAlition S Journal Checker Tool (JCT) ¹. I subsequently filtered the data based on the ESAC IDs of the selected 13 transformative agreements (see Table 1) to obtain information about journal portfolios and participating institutions pertaining specifically to the respective agreements. Journal information was further enriched with linking ISSNs (ISSN-Ls), as provided by the ISSN International Centre in the February 2026 version, based on ISSN matching to allow unique reference of journals.

Using the compiled JCT data, I retrieved article-level metadata from the Crossref April 2026 database snapshot and enriched it with first-author affiliation data from the OpenAlex Walden April 2026 database snapshot. All data was accessed via the SUB Göttingen Open Scholarly Data Warehouse. It should be noted that affiliation metadata coverage in OpenAlex has been reported to have declined substantially following the transition to Walden, partly due to publishers not sharing affiliation metadata via Crossref and partly due to technical issues within OpenAlex itself (Jahn 2025a), which may affect the completeness of first-author affiliation matching in this analysis.

The article-level data was restricted to the publication years 2019–2025, based on the issued date, to account for varying transformative agreement terms and full-year data availability (see Table 1). Non-scholarly journal content, such as table of contents etc., was excluded via paratext recognition, enrichment with first-author affiliation data was done by matching JCT participating institution information with OpenAlex affiliation data using ROR-IDs and institutional names as a fall back option. Furthermore the SOS Crossref truth table data from the January 2026 snapshot was matched using DOIs.

My workflow extends the current workflows for HOAD in the following respects: (1) journal coverage includes both hybrid and fully open access journals rather than hybrid journals only, (2) article data is restricted to pertain only to the selected 13 transformative agreements presented in Table 1, (3) I additionally enriched the data set with data from the SOS Crossref truth table.

To obtain information about metadata coverage, I computed counts of authors, affiliations, funders, and their respective persistent identifiers, as well as indicators for the presence of key metadata fields such as TDM support and open abstracts. Affiliation and ROR ID counts were computed as distinct values per article. This means that if multiple authors from the same institution appear on an article, the affiliation and its designated ROR ID are each counted only once. The number of affiliations was determined by first checking for ROR IDs and using name strings as a fall back option, when affiliations had no ROR IDs assigned. I identified open access articles by the presence of a CC licence in the version of record metadata. I therefore did not retrieve licence URLs for author accepted manuscript or TDM article versions, as identified by Crossref’s content_version field. Subsequently, articles without a CC licence were classified as non-open access, regardless of publisher policies or other access indicators.

The following SQL code shows the query used to retrieve the article-level data set, compute different metrics and join the SOS Crossref truth table.

WITH filtered_publications AS (
  SELECT DISTINCT
    UPPER(TRIM(cr.doi)) AS doi,
    jct.issn_l,
    jct.esac_id AS jn_esac_id,
    cr.publisher AS cr_publisher,
    cr.container_title AS cr_journal_title,
    EXTRACT(YEAR FROM cr.issued) AS cr_year,
    CASE WHEN cr.abstract IS NOT NULL THEN 1 END AS has_abstract,
    cr.license,
    cr.author,
    cr.link,
    cr.funder
  FROM (
      SELECT SPLIT(issn, ",") AS issn,
        doi,
        publisher,
        container_title,
        issued,
        license,
        abstract,
        author,
        link,
        funder
      FROM `subugoe-collaborative.cr_instant.snapshot`
      WHERE NOT REGEXP_CONTAINS(
          title,
          '(?i)^Author Index$|^Back Cover|^Contents$|^Contents:|^Corrigendum|^Cover Image|^Cover Picture|^Editorial Board|^Front Cover|^Frontispiece|^Inside Back Cover|^Inside Cover|^Inside Front Cover|^Issue Information|^List of contents|^Masthead|^Title page|^Correction$|^Corrections to|^Corrections$|^Withdrawn|^Frontmatter'
        )
        AND (
          NOT REGEXP_CONTAINS(page, '^S')
          OR page IS NULL
        )
        AND (
          NOT REGEXP_CONTAINS(issue, '^S')
          OR issue IS NULL
        )
        AND EXTRACT(YEAR FROM issued) BETWEEN 2019 AND 2025
  ) AS cr
  CROSS JOIN UNNEST(cr.issn) AS issn
  INNER JOIN `subugoe-collaborative.resources.oad_jct_jn` jct
    ON issn = jct.issn
  WHERE (
      (jct.esac_id = 'wiley2019deal' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2019 AND 2023) OR
      (jct.esac_id = 'wiley2024deal' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2024 AND 2025) OR
      (jct.esac_id = 'sn2021gac' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2021 AND 2023) OR
      (jct.esac_id = 'sn2020deal' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2020 AND 2023) OR
      (jct.esac_id = 'sn2024deal' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2024 AND 2025) OR
      (jct.esac_id = 'els2023deal' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2023 AND 2025) OR
      (jct.esac_id = 'degruy2022gac' AND EXTRACT(YEAR FROM cr.issued) = 2022) OR
      (jct.esac_id = 'degruy2023gac' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2023 AND 2024) OR
      (jct.esac_id = 'hogrefe2021gac' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2021 AND 2023) OR
      (jct.esac_id = 'hogrefe2024gac' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2024 AND 2025) OR
      (jct.esac_id = 'rsc2024tib' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2024 AND 2025) OR
      (jct.esac_id = 'opg2023tib' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2023 AND 2025) OR
      (jct.esac_id = 'ttp2024tib' AND EXTRACT(YEAR FROM cr.issued) BETWEEN 2024 AND 2025)
  )
),
first_author_data AS (
  SELECT
    UPPER(TRIM(w.doi)) AS doi,
    a.countries
  FROM `subugoe-collaborative.openalex_walden.works` w
  CROSS JOIN UNNEST(authorships) AS a
  WHERE a.author_position = 'first'
    AND 'DE' IN (
      SELECT UPPER(TRIM(country_code))
      FROM UNNEST(a.countries) AS country_code
    )
    AND EXISTS (
      SELECT 1
      FROM UNNEST(a.institutions) AS i
      JOIN `subugoe-collaborative.resources.oad_jct_inst` ji
        ON LOWER(i.ror) = LOWER(ji.ror_id)
        OR LOWER(i.display_name) = LOWER(ji.inst_name)
      WHERE i.display_name IS NOT NULL
        AND ji.inst_name IS NOT NULL
    )
),
oad_ta_md AS (
  SELECT
    fp.doi,
    fp.cr_year,
    fp.issn_l,
    fp.cr_journal_title,
    fp.jn_esac_id,
    fp.cr_publisher,
    fp.has_abstract,
    ARRAY_LENGTH(fp.author) AS num_authors,
    (SELECT COUNT(1) FROM UNNEST(fp.author) AS authors WHERE authors.orcid IS NOT NULL) AS num_orcids,
    COUNT(DISTINCT CASE WHEN md_2.id IS NOT NULL THEN md_2.id ELSE md_1.name END) AS num_affiliations,
    COUNT(DISTINCT CASE WHEN LOWER(md_2.id_type) = 'ror' THEN md_2.id END) AS num_rors,
    ARRAY_LENGTH(fp.funder) AS num_funders,
    (SELECT COUNT(1) FROM UNNEST(fp.funder) AS funders, UNNEST(funders.id) WHERE LOWER(id_type) = "doi") AS num_funder_dois,
    MAX(CASE WHEN md_3.content_version IN ('vor', 'unspecified') THEN md_3.url END) AS license_url,
    CASE
      WHEN EXISTS (SELECT 1 FROM UNNEST(fp.link) WHERE intended_application = "text-mining") THEN 1
      ELSE 0
    END AS has_tdm,
    tt.doi AS tt_doi,
    tt.count_authors,
    tt.has_authors_id_orcid,
    tt.count_authors_id_orcid,
    tt.has_affiliations,
    tt.count_affiliations,
    tt.has_affiliations_id_ror,
    tt.count_affiliations_id_ror,
    tt.has_abstract AS tt_has_abstract,
    tt.has_funders,
    tt.count_funders,
    tt.has_funders_id_doi,
    tt.count_funders_id_doi
  FROM filtered_publications fp
  INNER JOIN first_author_data fa
    ON fp.doi = fa.doi
  LEFT JOIN UNNEST(fp.author) AS md_0
  LEFT JOIN UNNEST(md_0.affiliation) AS md_1
  LEFT JOIN UNNEST(md_1.id) AS md_2
  LEFT JOIN UNNEST(fp.license) AS md_3
  LEFT JOIN `sos-datasources.truthtables.crossref_truthtable_20260131` tt
    ON fp.doi = tt.doi
  GROUP BY
    fp.doi,
    tt.doi,
    fp.cr_year,
    fp.issn_l,
    fp.cr_journal_title,
    fp.jn_esac_id,
    fp.cr_publisher,
    fp.has_abstract,
    fp.author,
    fp.funder,
    fp.license,
    fp.link,
    tt.count_authors,
    tt.has_authors_id_orcid,
    tt.count_authors_id_orcid,
    tt.has_affiliations,
    tt.count_affiliations,
    tt.has_affiliations_id_ror,
    tt.count_affiliations_id_ror,
    tt.has_abstract,
    tt.has_funders,
    tt.count_funders,
    tt.has_funders_id_doi,
    tt.count_funders_id_doi
)

SELECT * FROM oad_ta_md

Data cleaning and preparation involved several standardisation procedures: (1) harmonisation of ISSN-Ls for journals assigned to multiple ISSN-Ls, (2) consolidation of ESAC IDs for ISSN-Ls assigned to multiple ESAC IDs, in particular those resulting from publisher changes after the termination of the agreement, and (3) correction of errors in licence URLs. Journal-level standardisations were based on journal information from EZB and ZDB.

The so-compiled data set I used for the analysis consists of 246,499 articles published between 2019 and 2025 and enabled by one of the 13 transformative agreements of interest.

I used the SOS Crossref truth table data to validate my approach and found strong alignment for all coverage metrics (ORCIDs, funding information, funder DOIs, open abstracts) except for ROR coverage. Specifically, the ROR coverage values from the SOS dataset were higher than those I computed, in some instances even considerably. A more detailed analysis of the data revealed that these differences are attributable to 355 articles for which the absolute number of ROR IDs counted in the SOS dataset exceeds the absolute number of affiliations. Inspecting a sample of 20 DOIs from this set of articles via the Crossref API revealed that when a ROR ID was present for a given affiliation, no additional name string (Crossref field name) was included in the metadata. The SQL query used to create the SOS dataset and calculate the metrics shows that the number of affiliations is determined based on the name string in the Crossref affiliation name field. This differs from the calculation approach applied here. If publishers do not provide a name string for each affiliation, additionally to the respective ROR IDs, this explains why, for these articles, the number of ROR IDs is assessed as higher by the SOS counting method than the number of actual affiliations.

Results

The contractual provisions analysed in Dörner (2026), with underlying data provided in Dörner (2025), form the baseline for the analysis presented here. Briefly, the agreements with de Gruyter, Optica, and Trans Tech Publications mandated CC BY licensing; 12 of the 13 agreements committed to metadata delivery to Crossref, with several explicitly referencing ORCIDs, ROR IDs, and funding information as metadata fields available to the publishers; and 12 agreements contained TDM provisions.

Discussion

The analysis reveals substantial discrepancies between the contractual provisions of transformative agreements and their actual implementation in Crossref metadata, highlighting a gap in the operationalisation of open metadata commitments.

Outlook

One of the most compelling findings is the near-total absence of ROR ID coverage across most agreements. This underscores a clear opportunity for HOAD to expand its analytical scope by incorporating ROR coverage as a standard metric. Such an extension would be technically straightforward and would enable monitoring of ROR coverage across a wider set of agreements over time.

While this analysis is focused on 13 agreements with German consortia, the methodological approach could serve as a template for broader investigation. A distributed effort, connected to the Joint Task Force on Negotiating Openness of Publication Metadata between OA2020 (now OA Forward) and the Barcelona Declaration, has been started to analyse all openly available transformative agreements in the ESAC Registry of Open Publishing Agreements. The approach presented here could contribute to that initiative, providing empirical evidence to inform the task force’s work on negotiating openness of publication metadata across publishers and nations.

Data and Methods

The publicly accessible scholarly data warehouse, powered by Google BigQuery and operated by the Göttingen State and University Library, provides access to both an OpenAlex snapshot from September 2025 and the most recent Walden snapshot from February 2026.

The analysis was carried out by querying both OpenAlex versions separately and then comparing the results (i.e. no inner join, no shared dataset). This approach is intended to ensure that the new records added to Walden can also be included in the analysis. This is particularly the case for records labelled as expansion packs (which have an is_xpac=True in the data). The following analysis does not exclude retractions (is_retracted), paratexts (is_paratext) and the expansion pack (is_xpac). Usually, OpenAlex classifies records labelled with is_paratext=True as paratexts. However, there may occasionally be discrepancies, resulting in a record labelled is_paratext=True being classified as a normal article.

The analysis was conducted in Python. If you are interested in reproducing a figure, you may click on one of the grey arrows below to view my Python code, which was used to run the data analysis and generate the figures.

from google.cloud import bigquery
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
from matplotlib.lines import Line2D
from matplotlib.patches import Patch
import matplotlib.ticker as mtick
import matplotlib.image as mpimg

client = bigquery.Client(project='subugoe-collaborative')

openalex_snapshot = 'subugoe-collaborative.openalex.works'
walden_snapshot = 'subugoe-collaborative.openalex_walden.works'

sns.set_style('whitegrid')
plt.rc('font', family='Arial')
plt.rc('font', size=9) 
plt.rc('axes', titlesize=9) 
plt.rc('axes', labelsize=9) 
plt.rc('xtick', labelsize=9) 
plt.rc('ytick', labelsize=9) 
plt.rc('legend', fontsize=9)

def calculate_changes(df1_openalex, df2_walden, on):
    changes = pd.merge(df1_openalex, df2_walden, on=on, how='outer', suffixes=('_openalex', '_walden'))

    changes['n_openalex'] = changes['n_openalex'].fillna(0)
    changes['n_walden'] = changes['n_walden'].fillna(0)
    changes = changes[[on, 'n_openalex', 'n_walden']]

    changes['change'] = changes['n_walden'] - changes['n_openalex']
    changes['pct_change'] = (changes['n_walden'] - changes['n_openalex']) / changes['n_openalex'] * 100

    return changes

Summary and Outlook

I am aware that my analysis only covers a small part of the changes made in the Walden release. Even so, some useful information can be gained from this analysis. The Walden update demonstrates that OpenAlex is a constantly changing database. For example, there is a high degree of fluctuation in the classification of OA between the snapshots analysed with a large increase of Diamond OA combined with a decrease in other OA classes (especially Gold OA). On the other hand, improved coverage of references among larger publishers and better coverage of funding information can be observed. Concurrently, there is a slight decline in the publication volume of larger publishers (which is not unusual, because, for example, the publication year of an article is sometimes updated retrospectively).

Given these substantial changes, it is essential to report the version used, including timestamp and snapshot, when analysing OpenAlex to allow for comparability. In addition to using OpenAlex snapshots directly, there are several time-specific snapshots available on BigQuery through the ORION-DBs collective, which can improve the reproducibility of an analysis. Reporting changes as presented in this blog post can also help to interpret findings. This approach is, for instance, pursued by the German Kompetenzzentrum Bibliometrie, which publishes quality assurance reports on Zenodo for Scopus and Web of Science, and they plan to continue providing such reports for OpenAlex to illustrate changes between database snapshots to a wider audience.

Data and Code Availability

Python code for this data analysis is also available on GitHub: https://github.com/naustica/openalex_walden_analysis/.

Data was retrieved from the Data Warehouse of the SUB Göttingen: https://subugoe.github.io/scholcomm_analytics/data.html.

Data and methods

To better understand reports about declining affiliation metadata coverage in OpenAlex, and relating this to journal portfolios from major commercial publishers, the current journal portfolio of the Swedish BIBSAM consortium was examined based on journal data retrieved from the cOAlition S Journal Checker Tool as of 8 December 2025.

Affiliation coverage was compared across two OpenAlex sets: September 2025 snapshot and the most recent snapshot, to investigate whether the decrease is due to issues associated with the recent Walden release.

Both snapshots are publicly accessible via the SUB Göttingen Open Scholarly Data warehouse based on Google BigQuery. The following SQL query shows how the data was retrieved.

CREATE OR REPLACE TABLE `subugoe-collaborative.resources.au_data_check`
AS (
  SELECT DISTINCT
    oalex.doi,
    oalex.publication_year,
    oalex.primary_location.source.issn_l AS issn_l,
    jct_jn.esac_id AS esac_id,
    esac.publisher AS esac_publisher,
    oalex.primary_location.source.host_organization_name AS oalex_publisher,
    oalex.primary_location.source.is_in_doaj AS is_in_doaj,
    oalex.open_access.oa_status AS oa_status,
    au.author_position,
    CASE
      WHEN au.author IS NULL THEN 'Empty'
      ELSE 'Filled'
      END AS author_status,
    CASE
      WHEN ARRAY_LENGTH(au.institutions) = 0 THEN 'Empty'
      ELSE 'Filled'
      END AS institutions_status,
    CASE
      WHEN ARRAY_LENGTH(au.countries) = 0 THEN 'Empty'
      ELSE 'Filled'
      END AS countries_status,
    CASE
      WHEN au.is_corresponding IS NULL THEN 'Empty'
      ELSE 'Filled'
      END AS is_corresponding_status,
    CASE
      WHEN au.raw_author_name IS NULL THEN 'Empty'
      ELSE 'Filled'
      END AS raw_author_name_status,
    CASE
      WHEN ARRAY_LENGTH(au.raw_affiliation_strings) = 0 THEN 'Empty'
      ELSE 'Filled'
      END AS raw_affiliation_strings_status,
    CASE
      WHEN ARRAY_LENGTH(au.affiliations) = 0 THEN 'Empty'
      ELSE 'Filled'
      END AS affiliations_status,
    walden_au.author_position AS walden_author_position,
    CASE
      WHEN walden_au.author IS NULL THEN 'Empty'
      ELSE 'Filled'
      END AS walden_author_status,
    CASE
      WHEN ARRAY_LENGTH(walden_au.institutions) = 0 THEN 'Empty'
      ELSE 'Filled'
      END AS walden_institutions_status,
    CASE
      WHEN ARRAY_LENGTH(walden_au.countries) = 0 THEN 'Empty'
      ELSE 'Filled'
      END AS walden_countries_status,
    CASE
      WHEN walden_au.is_corresponding IS NULL THEN 'Empty'
      ELSE 'Filled'
      END AS walden_is_corresponding_status,
    CASE
      WHEN walden_au.raw_author_name IS NULL THEN 'Empty'
      ELSE 'Filled'
      END AS walden_raw_author_name_status,
    CASE
      WHEN ARRAY_LENGTH(walden_au.raw_affiliation_strings) = 0 THEN 'Empty'
      ELSE 'Filled'
      END AS walden_raw_affiliation_strings_status,
    CASE
      WHEN ARRAY_LENGTH(walden_au.affiliations) = 0 THEN 'Empty'
      ELSE 'Filled'
      END AS walden_affiliations_status
  FROM `subugoe-collaborative.openalex.works` AS oalex
  LEFT JOIN UNNEST(oalex.authorships) AS au
  LEFT JOIN UNNEST(au.countries) AS country
  INNER JOIN `subugoe-collaborative.openalex_walden.works` AS walden
    ON oalex.doi = walden.doi
  LEFT JOIN UNNEST(walden.authorships) AS walden_au
  INNER JOIN `subugoe-collaborative.oa2020.jct_jn` AS jct_jn
    ON oalex.primary_location.source.issn_l = jct_jn.issn_l
  INNER JOIN `subugoe-collaborative.oa2020.esac` AS esac
    ON jct_jn.esac_id = esac.id
  INNER JOIN
    `subugoe-collaborative.resources.document_classification_september25`
      AS doctype_classifier
    ON oalex.doi = doctype_classifier.doi
  WHERE
    (
      esac.organization = "Bibsam Consortium"
      AND end_date > '2024-12-31 00:00:00 UTC')
    AND oalex.is_paratext = FALSE
    AND is_research = TRUE
    AND (
      NOT REGEXP_CONTAINS(oalex.biblio.issue, '^[a-zA-Z]')
      OR oalex.biblio.issue IS NULL)
    AND (NOT (REGEXP_CONTAINS(oalex.title, '[0-9]{3} pp.')))
    AND oalex.publication_year BETWEEN 2018 AND 2025
)

To focus on original articles and reviews, paratext were excluded and results from the document type classifier provided by the German Competence Network for Bibliometrics (Haupka 2025) was used to refine the results. In order to check for potential data loading issues in Google BigQuery, a random sample of records was compared with the raw data dump files.

The resulting dataset can be found on Google BigQuery. After compiling the article-level data, absolute and relative coverage of author and affiliation metadata was calculated. Any data point that was not empty was counted as available.

SELECT 
  publication_year,
  COUNT(DISTINCT doi) AS total_dois,
  
  COUNT(DISTINCT CASE WHEN author_status = 'Filled' THEN doi END) AS with_author,
  COUNT(DISTINCT CASE WHEN institutions_status = 'Filled' THEN doi END) AS with_institutions,
  COUNT(DISTINCT CASE WHEN countries_status = 'Filled' THEN doi END) AS with_countries,
  COUNT(DISTINCT CASE WHEN is_corresponding_status = 'Filled' THEN doi END) AS with_is_corresponding,
  COUNT(DISTINCT CASE WHEN raw_author_name_status = 'Filled' THEN doi END) AS with_raw_author_name,
  COUNT(DISTINCT CASE WHEN raw_affiliation_strings_status = 'Filled' THEN doi END) AS with_raw_affiliation_strings,
  COUNT(DISTINCT CASE WHEN affiliations_status = 'Filled' THEN doi END) AS with_affiliations,
  
  COUNT(DISTINCT CASE WHEN walden_author_status = 'Filled' THEN doi END) AS with_walden_author,
  COUNT(DISTINCT CASE WHEN walden_institutions_status = 'Filled' THEN doi END) AS with_walden_institutions,
  COUNT(DISTINCT CASE WHEN walden_countries_status = 'Filled' THEN doi END) AS with_walden_countries,
  COUNT(DISTINCT CASE WHEN walden_is_corresponding_status = 'Filled' THEN doi END) AS with_walden_is_corresponding,
  COUNT(DISTINCT CASE WHEN walden_raw_author_name_status = 'Filled' THEN doi END) AS with_walden_raw_author_name,
  COUNT(DISTINCT CASE WHEN walden_raw_affiliation_strings_status = 'Filled' THEN doi END) AS with_walden_raw_affiliation_strings,
  COUNT(DISTINCT CASE WHEN walden_affiliations_status = 'Filled' THEN doi END) AS with_walden_affiliations

FROM `subugoe-collaborative.resources.au_data_check`
GROUP BY publication_year
ORDER BY publication_year

WITH doi_level_data AS (
  SELECT 
    doi,
    publication_year,
    MAX(CASE WHEN author_position IS NOT NULL THEN 1 ELSE 0 END) AS has_author_position,
    MAX(CASE WHEN author_status = 'Filled' THEN 1 ELSE 0 END) AS has_author,
    MAX(CASE WHEN institutions_status = 'Filled' THEN 1 ELSE 0 END) AS has_institutions,
    MAX(CASE WHEN countries_status = 'Filled' THEN 1 ELSE 0 END) AS has_countries,
    MAX(CASE WHEN is_corresponding_status = 'Filled' THEN 1 ELSE 0 END) AS has_is_corresponding,
    MAX(CASE WHEN raw_author_name_status = 'Filled' THEN 1 ELSE 0 END) AS has_raw_author_name,
    MAX(CASE WHEN raw_affiliation_strings_status = 'Filled' THEN 1 ELSE 0 END) AS has_raw_affiliation_strings,
    MAX(CASE WHEN affiliations_status = 'Filled' THEN 1 ELSE 0 END) AS has_affiliations,
    MAX(CASE WHEN walden_author_position IS NOT NULL THEN 1 ELSE 0 END) AS has_walden_author_position,
    MAX(CASE WHEN walden_author_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_author,
    MAX(CASE WHEN walden_institutions_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_institutions,
    MAX(CASE WHEN walden_countries_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_countries,
    MAX(CASE WHEN walden_is_corresponding_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_is_corresponding,
    MAX(CASE WHEN walden_raw_author_name_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_raw_author_name,
    MAX(CASE WHEN walden_raw_affiliation_strings_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_raw_affiliation_strings,
    MAX(CASE WHEN walden_affiliations_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_affiliations
  FROM `subugoe-collaborative.resources.au_data_check`
  GROUP BY doi, publication_year
)

SELECT 
  publication_year,
  COUNT(DISTINCT doi) AS total_dois,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_author_position = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS author_position_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_author = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS author_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_institutions = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS institutions_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_countries = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS countries_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_is_corresponding = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS is_corresponding_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_raw_author_name = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS raw_author_name_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_raw_affiliation_strings = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS raw_affiliation_strings_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_affiliations = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS affiliations_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_author_position = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_author_position_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_author = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_author_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_institutions = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_institutions_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_countries = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_countries_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_is_corresponding = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_is_corresponding_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_raw_author_name = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_raw_author_name_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_raw_affiliation_strings = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_raw_affiliation_strings_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_affiliations = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_affiliations_pct
FROM doi_level_data
GROUP BY publication_year
ORDER BY publication_year

SELECT 
  publication_year,
  esac_publisher,
  COUNT(DISTINCT doi) AS total_dois,
  
  COUNT(DISTINCT CASE WHEN author_status = 'Filled' THEN doi END) AS with_author,
  COUNT(DISTINCT CASE WHEN institutions_status = 'Filled' THEN doi END) AS with_institutions,
  COUNT(DISTINCT CASE WHEN countries_status = 'Filled' THEN doi END) AS with_countries,
  COUNT(DISTINCT CASE WHEN is_corresponding_status = 'Filled' THEN doi END) AS with_is_corresponding,
  COUNT(DISTINCT CASE WHEN raw_author_name_status = 'Filled' THEN doi END) AS with_raw_author_name,
  COUNT(DISTINCT CASE WHEN raw_affiliation_strings_status = 'Filled' THEN doi END) AS with_raw_affiliation_strings,
  COUNT(DISTINCT CASE WHEN affiliations_status = 'Filled' THEN doi END) AS with_affiliations,
  
  COUNT(DISTINCT CASE WHEN walden_author_status = 'Filled' THEN doi END) AS with_walden_author,
  COUNT(DISTINCT CASE WHEN walden_institutions_status = 'Filled' THEN doi END) AS with_walden_institutions,
  COUNT(DISTINCT CASE WHEN walden_countries_status = 'Filled' THEN doi END) AS with_walden_countries,
  COUNT(DISTINCT CASE WHEN walden_is_corresponding_status = 'Filled' THEN doi END) AS with_walden_is_corresponding,
  COUNT(DISTINCT CASE WHEN walden_raw_author_name_status = 'Filled' THEN doi END) AS with_walden_raw_author_name,
  COUNT(DISTINCT CASE WHEN walden_raw_affiliation_strings_status = 'Filled' THEN doi END) AS with_walden_raw_affiliation_strings,
  COUNT(DISTINCT CASE WHEN walden_affiliations_status = 'Filled' THEN doi END) AS with_walden_affiliations

FROM `subugoe-collaborative.resources.au_data_check`
GROUP BY publication_year, esac_publisher
ORDER BY publication_year, esac_publisher

WITH doi_level_data AS (
  SELECT 
    doi,
    publication_year,
    esac_publisher,
    MAX(CASE WHEN author_position IS NOT NULL THEN 1 ELSE 0 END) AS has_author_position,
    MAX(CASE WHEN author_status = 'Filled' THEN 1 ELSE 0 END) AS has_author,
    MAX(CASE WHEN institutions_status = 'Filled' THEN 1 ELSE 0 END) AS has_institutions,
    MAX(CASE WHEN countries_status = 'Filled' THEN 1 ELSE 0 END) AS has_countries,
    MAX(CASE WHEN is_corresponding_status = 'Filled' THEN 1 ELSE 0 END) AS has_is_corresponding,
    MAX(CASE WHEN raw_author_name_status = 'Filled' THEN 1 ELSE 0 END) AS has_raw_author_name,
    MAX(CASE WHEN raw_affiliation_strings_status = 'Filled' THEN 1 ELSE 0 END) AS has_raw_affiliation_strings,
    MAX(CASE WHEN affiliations_status = 'Filled' THEN 1 ELSE 0 END) AS has_affiliations,
    MAX(CASE WHEN walden_author_position IS NOT NULL THEN 1 ELSE 0 END) AS has_walden_author_position,
    MAX(CASE WHEN walden_author_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_author,
    MAX(CASE WHEN walden_institutions_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_institutions,
    MAX(CASE WHEN walden_countries_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_countries,
    MAX(CASE WHEN walden_is_corresponding_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_is_corresponding,
    MAX(CASE WHEN walden_raw_author_name_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_raw_author_name,
    MAX(CASE WHEN walden_raw_affiliation_strings_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_raw_affiliation_strings,
    MAX(CASE WHEN walden_affiliations_status = 'Filled' THEN 1 ELSE 0 END) AS has_walden_affiliations
  FROM `subugoe-collaborative.resources.au_data_check`
  GROUP BY doi, publication_year, esac_publisher
)

SELECT 
  publication_year,
  esac_publisher,
  COUNT(DISTINCT doi) AS total_dois,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_author_position = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS author_position_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_author = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS author_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_institutions = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS institutions_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_countries = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS countries_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_is_corresponding = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS is_corresponding_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_raw_author_name = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS raw_author_name_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_raw_affiliation_strings = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS raw_affiliation_strings_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_affiliations = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS affiliations_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_author_position = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_author_position_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_author = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_author_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_institutions = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_institutions_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_countries = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_countries_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_is_corresponding = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_is_corresponding_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_raw_author_name = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_raw_author_name_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_raw_affiliation_strings = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_raw_affiliation_strings_pct,
  ROUND(100.0 * COUNT(DISTINCT CASE WHEN has_walden_affiliations = 1 THEN doi END) / COUNT(DISTINCT doi), 2) AS walden_affiliations_pct
FROM doi_level_data
GROUP BY esac_publisher, publication_year
ORDER BY esac_publisher, publication_year

Summary tables were created for the entire dataset and by publisher journal portfolio for subsequent analyses. The resulting summary tables can be downloaded here:

All:

• au_md_all_absolute.csv
• au_md_all_rel.csv

By publisher

• au_md_by_publisher_absolute.csv
• au_md_by_publisher_rel.csv

Results

Overall, this investigation covers 13,462,902 journal articles published between 2018 and 2025 in journals covered by transformative agreements the BIBSAM consortium negotiated with 25 publishers, including the largest commercial publishers Elsevier, Springer Nature, and Wiley.

Figure 1 shows the affiliation metadata availability in OpenAlex by year and snapshot. Figure 1A highlights a massive decrease from 2024 onwards. While in 2023, 95% of records (n = 1,665,864) contained specific institution metadata, it decreased to 61% (n = 924,034), according to the September 2025 snapshot.

In Walden the decrease is even more pronounced: from 84% of records in 2023 (n = 1,475,406) containing specific institution metadata, it decreased to 54% (n = 816,018) in 2025. Overall, the Walden release contained 1,276,633 fewer articles with affiliation metadata than the previous snapshot analysing the same set of articles published between 2018 and 2025.

Comparing the availability of raw affiliation strings (Panel B) and author metadata (Panel C) shows that the decrease relates to availability of affiliation metadata, but does not extend to data about authors.

Looking ahead

To account for possible changes since the Walden snapshot release, 500 Elsevier articles published in 2025 without affiliation metadata were queried using the OpenAlex API on 15 December. The results show that around one-fourth of the articles now have affiliation metadata recorded in OpenAlex, so the situation is likely to change with the next release. However, this percentage is close to that reported for the September 2025 snapshot, which is still much lower than for previous years and only comprises a third of articles from this major publisher published in 2025.

Discussion and conclusion

Analysing the coverage of affiliation metadata in OpenAlex revealed a substantial decrease, presumably due to some publishers not providing affiliation metadata to Crossref, as well as potential technical issues within OpenAlex itself, which can partly be attributed to the transition to Walden. In particular, articles published in 2025 lacked affilation metadata in Crossref. Elsevier, the largest publisher in our sample, stands out in particular.

The observed decrease is critical for numerous institutional and country-specific research and monitoring activities making use of OpenAlex. In the case of the Hybrid Open Access Dashboard, I must alert users against relying on this data from 2024 onwards. I will carefully observe developments and hope that this issue can be resolved. Therefore, I have written this long-form issue description. Everyone is free to replicate these findings as the two latest OpenAlex snapshots used for this analysis are publicly available on Google BigQuery, as provided by the SUB Göttingen.

Data and Method

The following analysis is based on data from OpenAlex (Snapshot 08/2025) and Semantic Scholar (Snapshot 02/2025). Data was accessed via the data warehouse of the SUB Göttingen which is powered by Google BigQuery.

The present study focuses on articles and reviews in journals. For convenience, journal items were only filtered by the document and publication type from OpenAlex. In addition, the publication year from OpenAlex was used. Journal items were matched via the Digital Object Identifier (DOI). The analysis covers journal items from 2015 to 2023. To obtain the number of references from the respective data sources, the precomputed field was used (for example in Semantic Scholar: ‘referencecount’). However, sometimes the precalculated number may differ from the actual number of references contained in a database. For example, a journal article has a reference count of 18, but all references are missing in the database because the publisher decided to omit this information. In this case, I have to calculate the so-called source reference count.

The source reference count in Semantic Scholar was retrieved by counting items in the field ‘references’ that refer to other items within Semantic Scholar. Unlike the reference count, the actual references from Semantic Scholar are not contained in the data warehouse of the SUB Göttingen, but in an external dataset (available through the Semantic Scholar Datasets API). Because the Semantic Scholar reference dataset is quite large (about 1 TB), I made use of the High Performance Cluster of the GWDG Göttingen to calculate the number of source references. For this purpose, I wrote a Python script that performs this task. The aggregated data was then loaded into BigQuery. Retrieving the source reference count in OpenAlex is far simpler, as OpenAlex only considers the source reference count (field: ‘referenced_works_count’).

Results

Figure 1a displays a comparison of reference counts in Semantic Scholar and OpenAlex for the publication years 2015 to 2023, based on a shared corpus of approximately 37,5 million journal articles. Reference counts precalculated by Semantic Scholar are shown with a dashed line. It can be seen that the precalculated reference count in Semantic Scholar exceeds the number counted in OpenAlex for the publication years 2015 to 2019, but drops slightly afterwards. The number of source references in Semantic Scholar is lower across all publication years than in OpenAlex. In total, 982,585,826 references were counted in OpenAlex and 994,263,247 references were precalculated in Semantic Scholar. The number of source references in Semantic Scholar totals 896,752,043. Note that, although OpenAlex appears to have a greater number of source references than Semantic Scholar, I found about 9 million journal articles with more source references in Semantic Scholar than in OpenAlex (for the chosen time period).

In contrast to Figure 1a, Figure 1b shows a comparison of average reference counts in Semantic Scholar and OpenAlex. Again, reference counts precalculated by Semantic Scholar are shown with a dashed line. While the figure shows the same trends as in Figure 1a, a slight decline in the average reference figures can be observed from 2021 onwards. On average, OpenAlex counts 25.52 references per journal item, while Semantic Scholar counts 25.83 references and 23.29 source references per article.

In order to examine the differences between Semantic Scholar and OpenAlex in more detail, I will assign the precalculated reference and source reference count to publishers (see Figure 3a and 3b).

I also compiled a list of the most common publishers for which OpenAlex or Semantic Scholar has a larger source reference count (see Table 1 and 2). Here, it can be seen that OpenAlex deposits more source references for the publishers Elsevier (about 12,600,000 references), Springer (about 916,000 references) and Wiley (about 746,000 references) than Semantic Scholar. On the contrary, Semantic Scholar has more source references for the publishers Frontiers Media (about 800,000 references), SAGE (about 512,000 references) and Taylor & Francis (about 269,000 references).

Finally, I will compare the reference and source reference count in Semantic Scholar and OpenAlex by using topics from OpenAlex (see Table 3 and Table 4). I choose the primary topic domain for this purpose (which comprises the domains Health Sciences, Social Sciences, Life Sciences and Physical Sciences). For health sciences, Semantic Scholar counts on average 22,4 source references per article, while OpenAlex counts 23,9 source references. This differs slightly from the precomputed references count, where Semantic Scholar exceeds the reference count of OpenAlex by 24,1 to 23,9 references. The same applies to the social sciences, where the number of references precalculated by Semantic Scholar is greater than in the case of OpenAlex. However, when looking at the average number of source references, OpenAlex surpasses Semantic Scholar (16,0 to 14,5 source references on average).

Conclusion

This analysis demonstrates interesting differences between the reference counts in OpenAlex and Semantic Scholar. In particular, discrepancies were found between publishers, with Semantic Scholar counting more references than OpenAlex for publishers like Taylor & Francis and BMJ. The proportion of source references is higher in OpenAlex than in Semantic Scholar, although about 9 million journal articles were identified for which the source reference count was greater than in OpenAlex. This finding is quite relevant for bibliometric research, as it may enable more accurate citation analysis in OpenAlex, which can be achieved through the inclusion of open metadata.

During my analysis, I also discovered some inconsistencies between the Semantic Scholar website and its API. For example, references for some journal articles are missing in the API but are shown on the website. In other cases, references are missing in the API with the note: “Notice: The following paper fields have been elided by the publisher: {‘references’}”. However, this observation was not analysed further.

Data and Code Availability

The source code for the underlying analysis can be found on GitHub: https://github.com/naustica/oal_s2_ref/.

Data is available in the Data Warehouse of the SUB Göttingen: https://subugoe.github.io/scholcomm_analytics/data.html.