Cluster1 sources· last seen 5h ago· first seen 5h ago
[R] PCA on ~40k × 40k matrix in representation learning — sklearn SVD crashes even with 128GB RAM. Any practical solutions?
Hi all, I'm doing ML research in representation learning and ran into a computational issue while computing PCA. My pipeline produces a feature representation where the covariance matrix A^TA is roughly 40k × 40k. I need the full eigendecomposition / PCA basis, not just the top-k components. Cur
Lead: r/MachineLearningBigness: 22pca40kmatrixrepresentationsklearn
📡 Coverage
10
1 news source
🟠 Hacker News
0
🔴 Reddit
50
18 upvotes across 1 sub
📈 Google Trends
0
Full methodology: How scoring works
Receipts (all sources)
[R] PCA on ~40k × 40k matrix in representation learning — sklearn SVD crashes even with 128GB RAM. Any practical solutions?
REDDIT · r/MachineLearning · 5h ago · ⬆ 18 · 💬 57
score 113
Hi all, I'm doing ML research in representation learning and ran into a computational issue while computing PCA. My pipeline produces a feature representation where the covariance matrix A^TA is roughly 40k × 40k. I need the full eigendecomposition / PCA basis, not just the top-k components. Cur