LipidSearch 5.2: Improved Software for Parallel Processing of Large Datasets from LC-MS High Resolution Mass Spectrometry Based Lipidomics Workflows

Importance of the topic

Lipidomics workflows based on LC-high resolution mass spectrometry (LC-HRMS/MS) generate very large and structurally complex datasets. Efficient, reliable software for peak detection, MS2 deconvolution, lipid identification and cross-sample alignment is essential to convert raw spectral data into reproducible lipid annotations and quantitative results. Improvements in computation strategy directly impact throughput for large cohort studies, QA/QC work, and discovery projects where hundreds of files and gigabytes of data must be processed without loss of fidelity.

Objectives and overview of the study

The authors describe the development and performance validation of LipidSearch software version 5.2 (and subsequent 5.2.x builds), focusing on parallel processing strategies, memory and cache management, and practical hardware recommendations. The work compares processing times across software versions (5.1.x → 5.2.2.1), evaluates performance on several representative LC-HRMS/MS datasets of varying sizes (from single-file QC to large multi-sample cohorts including a 145 GB P250 dataset), and reports best practices for system configuration and acquisition workflows that minimize redundant MS2 searching.

Methodology

Key algorithmic and workflow changes implemented in LipidSearch 5.2:

• Two-stage processing architecture: (1) peak extraction and MS2 deconvolution executed in parallel over segmented time regions of the chromatogram (BPC, MS1 peak extraction, EIC extraction, MS2 deconvolution); (2) lipid identification and alignment executed as parallel searches grouped by m/z and by lipid class across samples.
• Parallelization across both time axis segments and m/z groups to exploit multi-core CPU resources while avoiding excessive memory allocation.
• Active RAM and cache management: realtime monitoring of parallel processes, deferred allocation and intelligent caching of intermediate results to avoid memory overloads and frequent allocation/deallocation penalties.
• Support for workflows that leverage AcquireX MS2 (iterative exclusion / pooled MS2) to reduce redundant MS2 acquisition and substantially cut search overhead compared to naive ddMS2 per injection.

Representative acquisition and processing settings used in performance tests:

• Mass tolerances: typical MS / MS2 tolerances tested were 3 ppm / 5 ppm (high-resolution conditions) and 5 ppm / 10 ppm for other datasets.
• Retention time tolerances for alignment ranged from 0.10–0.25 min depending on dataset.
• LC conditions: 25–60 min reversed-phase separations on C18, C30, and prototype columns; instruments tested included Orbitrap Exploris 240, Q Exactive series, Q Exactive HF, and Fusion Lumos with ddMS2 acquisition.

Instrumentation used

Mass spectrometers and chromatography systems used for test datasets:

• Thermo Fisher Orbitrap Exploris 240
• Thermo Fisher Q Exactive and Q Exactive HF
• Thermo Fisher Fusion Lumos
• Columns: Ascentis C18, Acclaim C30, Accucore C30, prototype C30 columns

Workstations and storage used for benchmarking:

• System 1 (Z840): 2× Intel E5-2667 (8 cores each), 128 GB RAM, 4 TB SATA SSD (870-QVO)
• System 2 (Z8G4): 2× Intel Gold 6148 (20 cores each), 384 GB RAM, 4 TB NVMe (990 PRO M.2)
• System 3 (Lenovo): Intel i9-12900 (16 cores; 8 P + 8 E), 128 GB RAM, 4 TB NVMe (XG8 M.2)

Main results and discussion

Performance improvements and observations:

• Version advances from 5.1.x to 5.2.2.1 produced substantial speedups for both search and alignment steps, with average total processing time reductions on the order of 5-fold across test datasets.
• Parallelization of peak detection, MS2 deconvolution, lipid searching and alignment combined with improved memory/caching produced much faster throughput, especially on multi-socket, many-core systems with fast NVMe storage.
• For very large alignment jobs (datasets >100 files, e.g., the 145 GB P250 dataset), system configuration impacted which step was limiting: Z8G4 (System 2) yielded best overall search performance, while the older Z840 performed best for alignment on the largest dataset in some cases—highlighting the complex interactions among core count, single-thread speed, memory capacity and I/O.
• Adopting AcquireX-style MS2 acquisition (pooled iterative exclusion) reduced redundant MS2 searching and yielded roughly a 5× reduction in processing overhead relative to acquiring ddMS2 per sample.
• Memory and cache management prevented failures and excessive slowdowns previously seen when processing large (>100-file) cohorts, allowing completion of alignment runs that formerly failed to finish.

Quantitative highlights from representative datasets:

• Processing times scaled with dataset size (GB) and instrument acquisition strategy; the P250 (145 GB) dataset saw the most marked absolute improvements.
• Smaller datasets (e.g., FB1 mouse liver, SRM1950 QC) completed in minutes to a few hours depending on system; large cohort datasets completed in substantially less time on 5.2.x compared to 5.1.x builds.

Benefits and practical applications

• Dramatically reduced processing time enables routine processing of large lipidomics cohorts and public reference datasets without sacrificing annotation quality.
• Improved stability and memory usage reduces risk of crashed or stalled alignment jobs for multi-sample studies.
• Recommendations on hardware (high single-core clock speed, many cores, >128 GB RAM, NVMe storage) give laboratories guidance to cost-effectively optimize throughput.
• Integration with AcquireX-style acquisition strategies lowers redundant data and further improves throughput—useful for studies where pooled MS2 acquisition is feasible.

Future trends and opportunities

Potential directions and uses arising from these improvements:

• Further parallel and distributed processing: scaling LipidSearch to cluster or cloud environments could allow near-linear speedups for very large cohorts and centralized lipidomics facilities.
• Hybrid workflows combining smart acquisition (AcquireX) with on-the-fly identification and QC could shorten time from data acquisition to interpretable results.
• Continued optimization of memory/caching strategies and use of persistent intermediate-format files could reduce re-processing times for iterative analyses.
• Integration with downstream statistical and pathway analysis platforms to streamline end-to-end lipidomics pipelines for biomarker discovery and regulatory QA/QC.

Conclusion

LipidSearch 5.2 and subsequent 5.2.x builds demonstrate meaningful, validated improvements in processing speed, stability and scalability for LC-HRMS/MS-based lipidomics. Parallelized algorithms across time regions and m/z groups, together with careful memory and cache management, enable routine processing of large datasets that were previously slow or prone to failure. Combined with recommended hardware (NVMe storage, ≥128 GB RAM, high single-core frequency and many cores) and acquisition strategies that reduce redundant MS2, these software updates substantially increase laboratory throughput and reduce computational bottlenecks.

References

• Peake DA, Kitahashi Y, Masaki N, Yokoi Y. LipidSearch 5.2: Improved software for parallel processing of large datasets from LC-MS high resolution mass spectrometry based lipidomics workflows. Peake Performance LLC and Mitsui Knowledge Industry Co.; manuscript reporting benchmark testing and beta evaluations.