Linq Query efficiency

The main issue is likely not the initial GroupBy. LINQ can have performance costs, but the expensive part here is the repeated filtering inside the foreach, especially this pattern:

InitialData.AsQueryable().Where(CombFilter).ToList()

That re-runs a query for every item in lstRankRangeSumRankPartition, and the dynamic string-based predicate adds extra overhead.

A more efficient approach is to group or index InitialData once using the same composite key used in the first query, then look up matching subsets instead of scanning repeatedly.

Example shape:

var groupedInitialData = InitialData
    .GroupBy(x => (
        x.RankRange0Count,
        x.RankRange1Count,
        x.RankRange2Count,
        x.RankRange3Count,
        x.RankRange4Count,
        x.TotalFrequency,
        x.TotalSum))
    .ToDictionary(g => g.Key, g => g.ToList());

var lstRankRangeSumRankPartition = lstAllCombs
    .GroupBy(p => (
        p.RankRange0Count,
        p.RankRange1Count,
        p.RankRange2Count,
        p.RankRange3Count,
        p.RankRange4Count,
        p.TotalSum,
        p.TotalFrequency))
    .Select(group => new SpecLevel
    {
        RankRange0Count = group.Key.RankRange0Count,
        RankRange1Count = group.Key.RankRange1Count,
        RankRange2Count = group.Key.RankRange2Count,
        RankRange3Count = group.Key.RankRange3Count,
        RankRange4Count = group.Key.RankRange4Count,
        RankSum = group.Key.TotalFrequency,
        TotalSum = group.Key.TotalSum,
        Counter = group.Count()
    })
    .ToList();

lstRankRangeSumRankPartition = UpdateListByPercentage(lstRankRangeSumRankPartition);

foreach (var itemSpecs in lstRankRangeSumRankPartition)
{
    if (itemSpecs.Percentage > 50)
        break;

    var key = (
        itemSpecs.RankRange0Count,
        itemSpecs.RankRange1Count,
        itemSpecs.RankRange2Count,
        itemSpecs.RankRange3Count,
        itemSpecs.RankRange4Count,
        itemSpecs.RankSum,
        itemSpecs.TotalSum);

    if (groupedInitialData.TryGetValue(key, out var filteredSubset))
    {
        FilteredData.AddRange(filteredSubset);
    }
}

Why this is better:

1. InitialData is grouped once instead of filtered repeatedly.
2. ToDictionary caches the grouped results, which matches the documented LINQ optimization pattern of materializing results when repeated execution would otherwise occur.
3. It avoids repeated query execution caused by deferred evaluation.
4. It removes the overhead of string-based dynamic filtering.

Additional notes:

• If the same query results are used repeatedly, materializing with ToList() or ToArray() can reduce repeated execution overhead.
• LINQ and lambda-heavy code can have noticeable performance costs in hot paths.
• If this code still runs frequently, measure whether replacing some LINQ with direct loops improves throughput.

If duplicates in FilteredData are not wanted, add a deduplication step because multiple SpecLevel items could map to overlapping results depending on the data model.

References:

• Work with Language-Integrated Query (LINQ)
• LINQ queries
• Writing Large, Responsive .NET Framework Apps