---
jupyter:
  jupytext:
    text_representation:
      extension: .Rmd
      format_name: rmarkdown
      format_version: '1.2'
  kernelspec:
    display_name: R
    language: R
    name: ir
---

# pRoloc integration: for pRoloc users

The [grassp data portal](https://grassp.apps.czbiohub.org/datasets) publishes over a hundred
curated subcellular fractionation datasets, uniformly processed, each with several published
marker sets already mapped onto it. They are stored as `.h5ad`, which sounds like a Python
format, but the `grasspio` R package (uses `anndataR` under the hood) can translate it to an `MSnSet`.
This means that you can use pRoloc to analyze and visualize a grassp dataset without ever leaving R.

So this tutorial needs **no Python at all**. You download a file, get an
`MSnSet`, and from there it is ordinary pRoloc. It also goes the other way: the last section
takes a `pRolocdata` object and writes it out as h5ad.

```{note}
This tutorial is for someone who works primarily in **R** and wants to use grassp's data or a
grassp method. **If you work primarily in Python, read the companion tutorial instead:**
{doc}`pRoloc integration: for grassp users <notebooks/proloc_tutorial>`.
```

## Requirements

```r
install.packages(c("remotes", "BiocManager"))

# We need rdf5 to read the on disk h5ad format that grassp uses, and pRoloc/pRolocdata for the processing.
BiocManager::install(c("rhdf5", "pRoloc", "pRolocdata"))

# grasspio is a small R package that grassp provides to read/write h5ad files. It uses scVerse's anndataR package under the hood.
remotes::install_github("czbiohub-sf/grassp", subdir = "r/grasspio")
```

Note that `grasspio` requires R >=4.5.

```{r}
suppressPackageStartupMessages({
  library(grasspio)
  library(pRoloc)
  library(MSnbase)
})
# IRkernel decides figure size from these rather than from knitr chunk options.
options(repr.plot.width = 9, repr.plot.height = 6, width = 100)
# Print results as an R console would, rather than as IRkernel's rich HTML. 
options(jupyter.rich_display = FALSE)
```

## Getting a dataset

Browse [the portal](https://grassp.apps.czbiohub.org/datasets), pick a dataset, and take its
`.h5ad`. The files sit at a predictable URL, so you can download one in your browser or fetch it
from R:

```
https://public.czbiohub.org/proteinxlocation/datasets/<name>.h5ad       # processed
https://public.czbiohub.org/proteinxlocation/datasets_raw/<name>.h5ad   # pre-enrichment
```

The `datasets/` objects are already preprocessed: enriched, replicates collapsed, and each
protein's profile already normalised to sum to 1. The `datasets_raw/` objects are raw intensities
with replicates preserved — useful, but they need normalising before pRoloc's distance-based
methods mean anything.

Here we take `Currie_2024_AC16_Control`, a LOPIT-DC experiment on AC16 cardiomyocytes for demonstration.:

```{r}
h5ad_path <- file.path(tempdir(), "Currie_2024_AC16_Control.h5ad")
download.file(
  "https://public.czbiohub.org/proteinxlocation/datasets/Currie_2024_AC16_Control.h5ad",
  h5ad_path,
  mode = "wb"
)
```

```{r}
x <- grassp_as_msnset(h5ad_path, nan_to_unknown = TRUE)
x
```

That is the whole conversion. There is nothing special about the file: a portal dataset is a plain
h5ad (see the [AnnData](https://anndata.scverse.org/en/stable/) documentation for more info). The one thing `grassp_as_msnset()` does change is the
`nan_to_unknown` default, which rewrites `NA` markers to the literal `"unknown"` that pRoloc
assumes, because `markerMSnSet()` fails outright on `NA`.

## Getting oriented

Everything in the AnnData object has landed somewhere, slot for slot:

| grassp `AnnData` | pRoloc `MSnSet` |
| --- | --- |
| `.X` | `exprs()` which is the first `assayData` slot|
| `.obs_names` / `.var_names` | `featureNames()` / `sampleNames()` |
| `.obs` | `fData()` scalar columns |
| `.obsm[k]`, as a DataFrame | `fData()[[k]]` — a **matrix nested inside one column** |
| `.var` | `pData()` |
| `.varm[k]`, as a DataFrame | `pData()[[k]]` — the same thing on the sample axis |
| `.layers[k]` | extra `assayData` elements |
| `.uns["processing"]` | `processingData()@processing` |
| everything else in `.uns` | `experimentData()@other$grassp_uns` |
| `.obsp` / `.varp` | **nothing** — `eSet` has no pairwise slot |

So per-protein annotations are `fData`, per-fraction ones are `pData`, and the extra matrices are
additional `assayData` elements:

```{r}
dim(x)
head(fvarLabels(x), 12)
assayDataElementNames(x)
```

Some `fData` columns are themselves **matrices** — the per-protein × per-compartment arrays that
came from AnnData's `obsm`. That is pRoloc's own idiom for score matrices, so they arrive in the
shape pRoloc expects. Note that grassp stores PCA and UMAP coordinates in `fData(x)$X_pca`. This 
means that plotting can be done cheaply without computing the dimensionality reduction again.

```{r}
vapply(
  fData(x),
  function(col) if (is.matrix(col)) paste(dim(col), collapse = " x ") else "",
  character(1)
)[c("X_pca", "X_umap", "harmonized_annotation_propagated_probabilities")]
```

The publication metadata came too. An `MSnSet` has no slot for arbitrary key-value metadata, so
`grasspio` parks whatever it cannot map on `experimentData(x)@other$grassp_uns`:

```{r}
uns <- experimentData(x)@other$grassp_uns
unlist(uns[c("title", "publication_journal", "publication_doi")])
```

### Pick a marker column — there is no single one

The portal maps several published marker sets onto every dataset. They are the marker lists from
the papers themselves, matched onto each dataset's protein IDs by grassp's
`pp.add_markers()`, many of them imported from `pRoloc::pRolocmarkers()`

We can check which marker columns are present, how many classes each has, and how many proteins are labelled in each:

```{r}
marker_cols <- grep("^marker_|^author_markers", fvarLabels(x), value = TRUE)
data.frame(
  column = marker_cols,
  classes = vapply(marker_cols, function(f) length(getMarkerClasses(x, fcol = f)), integer(1)),
  labelled = vapply(marker_cols, function(f) sum(fData(x)[[f]] != "unknown"), integer(1)),
  row.names = NULL
)
```

We will use `marker_lilley`, the set that grassp imports from `pRoloc::pRolocmarkers()`:

```{r}
FCOL <- "marker_lilley"
getMarkerClasses(x, fcol = FCOL)
dim(markerMSnSet(x, fcol = FCOL))
c(missing = sum(is.na(exprs(x))), rows_summing_to_1 = sum(abs(rowSums(exprs(x)) - 1) < 1e-6))
```

No preprocessing is needed on an enriched object: `filterNA()` and `normalise()` could be applied here if you load a raw object.

**One trap before you plot.** pRoloc defaults to `fcol = "markers"`, and a portal object has no
such column — so a bare `plot2D(x)` errors with `fcol %in% fvarLabels(object) is not TRUE`.
Always name your column, or pass `fcol = NULL`.

## Plotting

```{r}
plot2D(x, fcol = FCOL, main = "pRoloc default PCA (scaled)")
addLegend(x, fcol = FCOL, where = "topright", cex = 0.6, ncol = 2)
```

```{r}
options(repr.plot.height = 4)
plot2D(x, method = "scree")
options(repr.plot.height = 6)
```

## Comparing pRoloc's PCA with the portal's own

The portal object already carries PCA coordinates in `fData(x)$X_pca`, computed by grassp when
the dataset was built. Recomputing them in pRoloc is a good way to check you are looking at the
same object — but only if you match the convention, and **pRoloc's default does not**:

```{r}
stored <- fData(x)$X_pca
colnames(stored) <- paste0("PC", seq_len(ncol(stored)))

default_pca <- plot2D(x, fcol = FCOL, plot = FALSE)
colnames(default_pca) # pRoloc prints the variance explained into the names
c(PC1 = cor(default_pca[, 1], stored[, 1]), PC2 = cor(default_pca[, 2], stored[, 2]))
```

PC2 correlates at about 0.24. Nothing is broken: `plot2D()` passes `scale = TRUE` to `prcomp`,
scaling every fraction to unit variance, while the PCA in the grassp object was centered but not scaled. Scaling does not
merely stretch the axes — it changes the eigenvectors *and their order*, so comparing
component-by-component across the two conventions compares different directions.

Match the convention and they agree:

```{r}
matched <- plot2D(x, fcol = FCOL, plot = FALSE, methargs = list(center = TRUE, scale = FALSE))
colnames(matched)
vapply(1:2, function(k) cor(matched[, k], stored[, k]), numeric(1))
```


`cor(PC2) = -1` is not a disagreement either. An eigenvector is only defined up to sign, so some
components come back negated; that is arithmetic, not a discrepancy. Fix it explicitly:

### Drawing grassp's embeddings with pRoloc

`plot2D(method = "none")` takes coordinates you already have and styles them like any other
pRoloc plot. That works for the stored PCA and, usefully, for the UMAP — which spares you
compiling a UMAP package just to look at it:

```{r}
umap <- fData(x)$X_umap
colnames(umap) <- c("UMAP1", "UMAP2")
par(mfrow = c(1, 2))
plot2D(stored[, 1:2], method = "none", methargs = list(x), fcol = FCOL, main = "portal X_pca")
plot2D(umap, method = "none", methargs = list(x), fcol = FCOL, main = "portal X_umap")
par(mfrow = c(1, 1))
```

## Profiles and marker quality

`plotDist()` draws one line per protein across the fractions, and `markers =` picks out a subset
to colour over the rest in grey. Passing the whole object and one compartment's markers therefore
shows that compartment's profile against the backdrop of everything measured — how distinctive
the profile is, and how tightly the markers agree with each other:

```{r}
par(mfrow = c(1, 2))
for (cl in c("Mitochondrion", "ER")) {
  markers <- featureNames(x)[fData(x)[[FCOL]] == cl]
  plotDist(x, markers = markers, pch = 1,
           main = sprintf("%s (%d markers)", cl, length(markers)))
}
par(mfrow = c(1, 1))
```

```{r}
options(repr.plot.height = 5)
plotConsProfiles(mrkConsProfiles(x, fcol = FCOL))
options(repr.plot.height = 6)
```

`QSep()` quantifies how well separated the marker classes are — the single most useful sanity
check on a fractionation experiment, and something the portal makes unusually easy because you
can ask it of every marker set at once. Higher is better: it is the between-class distance in
units of within-class spread, so a value near 1 means a class is no further from its neighbours
than from itself.

```{r}
summary(QSep(x, fcol = FCOL))
sapply(marker_cols, function(f) median(summary(QSep(x, fcol = f), verbose = FALSE)))
```

## Classification

`svmClassification()` needs hyperparameters. A real analysis gets them from `svmOptimisation()`,
which is a grid search over repeated stratified cross-validation and takes minutes, so it is
shown here but not run:

```r
params <- svmOptimisation(x, fcol = FCOL, times = 100,
                          class.weights = classWeights(x, fcol = FCOL))
plot(params)
getParams(params)
```

```{r}
set.seed(1) # e1071's probability scaling consumes the RNG; without this, runs differ
x <- svmClassification(x, fcol = FCOL, sigma = 0.1, cost = 16, scores = "all")

# `scores = "all"` writes the per-class matrix but NOT the scalar score that orgQuants() and
# getPredictions() look for, so derive it.
fData(x)$svm.scores <- apply(fData(x)$svm.all.scores, 1, max)
dim(fData(x)$svm.all.scores)
```

The output columns are named `svm`, `svm.scores`, `svm.all.scores` regardless of which `fcol` you
trained on. The bare `[1] "marker_lilley"` above the result is a stray `print()` inside
`MLInterfaces`, not something you did.

```{r}
# `mcol` is the second fcol-like trap: orgQuants() and getPredictions() default it to "markers"
# too, and use it to decide which proteins were training data.
ts <- orgQuants(x, fcol = "svm", scol = "svm.scores", mcol = FCOL, t = 0.75, verbose = FALSE)
ts[is.na(ts)] <- Inf # a class with too few markers has no quantile
x <- getPredictions(x, fcol = "svm", scol = "svm.scores", mcol = FCOL, t = ts, verbose = FALSE)
table(fData(x)$svm.pred == "unknown")
```

```{r}
plot2D(x, fcol = "svm.pred", methargs = list(center = TRUE, scale = FALSE),
       main = "SVM predictions, thresholded")
```

A second classifier is a cheap sanity check, and `QSep` closes the loop quantitatively — do the
predicted labels separate better than the markers you started from?

```{r}
set.seed(1)
x <- knnClassification(x, fcol = FCOL, k = 5) # note: knnClassification rejects `verbose`
c(svm_vs_knn_agreement = mean(fData(x)$svm == fData(x)$knn))
c(markers = median(summary(QSep(x, fcol = FCOL), verbose = FALSE)),
  svm.pred = median(summary(QSep(x, fcol = "svm.pred"), verbose = FALSE)))
```

## Going the other way: a native pRoloc object out to h5ad

The bridge is symmetric, and useful even if you never touch Python — neither pRoloc nor MSnbase
ships an h5ad exporter. Any `MSnSet` will do, including the ones `pRolocdata` distributes:

```{r}
suppressPackageStartupMessages(library(pRolocdata))
data(dunkley2006)
dunkley2006
```

```{r}
out <- file.path(tempdir(), "dunkley2006.h5ad")
grassp_write_msnset(dunkley2006, out, overwrite = TRUE)
```

Scalar `fData` columns become `obs`, matrix-valued ones become `obsm` data frames that carry their
own class names, extra `assayData` elements become layers, and `"unknown"` becomes `NA` on
the way out because that is grassp's encoding. Reading it back gives the same object:

```{r}
back <- grassp_as_msnset(out)
c(
  same_dim = identical(dim(back), dim(dunkley2006)),
  exprs_equal = isTRUE(all.equal(unname(exprs(back)), unname(exprs(dunkley2006)),
                                 tolerance = 1e-8)),
  same_columns = identical(sort(fvarLabels(back)), sort(fvarLabels(dunkley2006)))
)
getMarkerClasses(back, fcol = "markers")
```

That file is what a grassp user would open with `anndata.read_h5ad("dunkley2006.h5ad")` — no
grassp function required, because it is an ordinary h5ad. The mapping table, and the few things
that cannot cross, are in `vignette("grasspio")` and in {doc}`../api/io`.

## Session info

```{r}
sessionInfo()
```
