Splicing tests

[bschilder]

I'm testing out the splicing branch #38 with a GTF file:

Download

# From Ensembl FTP: https://ftp.ensembl.org/pub/release-113/gtf/homo_sapiens/
gtf_path = pooch.retrieve(
    url="https://ftp.ensembl.org/pub/release-113/gtf/homo_sapiens/Homo_sapiens.GRCh38.113.chr_patch_hapl_scaff.gtf.gz",
    known_hash="670d0572422b315124770d1dff1613ff5b563b965de48d8f35cccdef892bfc35",
    progressbar=True,
    fname="Homo_sapiens.GRCh38.113.chr_patch_hapl_scaff.gtf.gz",
)

Read in the GTF with gtfparse

import gtfparse

gtf = gtfparse.read_gtf(
    gtf_path,
    # os.path.expanduser("~/projects/ProHap/data/gtf/Homo_sapiens.GRCh38.113.chr_patch_hapl_scaff_chr10.gtf"),
    features=["exon"]
    )

Subset GTF and convert col names:

chr22_genes = (
    gtf
    .filter(pl.col("seqname") == "22")
    .filter(pl.col("feature") == "exon")
    .rename({"seqname": "chrom", "start": "chromStart", "end": "chromEnd"})
)
chr22_genes.head()

Download VCF

# From 1KG FTP: https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working/20220422_3202_phased_SNV_INDEL_SV/
vcf_path = pooch.retrieve(
    url="https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working/20220422_3202_phased_SNV_INDEL_SV/1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.vcf.gz",
    known_hash="91aeaf0d1e805cf4223c5cbf12e22afc28db1709054c42bd668337219f8fddcc",
    progressbar=True,
    fname="1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.vcf.gz",
)

Create GVL database

gvl.write(
    path="1kg_chr22.gvl",
    bed=chr22_genes,
    variants=vcf_path,
    max_jitter=0,
    overwrite=True
)

Error:

---------------------------------------------------------------------------
ComputeError                              Traceback (most recent call last)
Cell In[31], [line 1](vscode-notebook-cell:?execution_count=31&line=1)
----> [1](vscode-notebook-cell:?execution_count=31&line=1) gvl.write(
      [2](vscode-notebook-cell:?execution_count=31&line=2)     path="1kg_chr22.gvl",
      [3](vscode-notebook-cell:?execution_count=31&line=3)     bed=chr22_genes,
      [4](vscode-notebook-cell:?execution_count=31&line=4)     variants=vcf_path,
      [5](vscode-notebook-cell:?execution_count=31&line=5)     max_jitter=0,
      [6](vscode-notebook-cell:?execution_count=31&line=6)     overwrite=True
      [7](vscode-notebook-cell:?execution_count=31&line=7) )

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:108, in write(path, bed, variants, bigwigs, samples, max_jitter, overwrite, max_mem, phased, ccf_field)
    [105](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:105) if isinstance(bed, (str, Path)):
    [106](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:106)     bed = read_bedlike(bed)
--> [108](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:108) gvl_bed, contigs, input_to_sorted_idx_map = _prep_bed(bed, max_jitter)
    [109](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:109) bed.with_columns(r_idx_map=pl.lit(input_to_sorted_idx_map)).write_ipc(
    [110](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:110)     path / "input_regions.arrow"
    [111](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:111) )
    [112](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:112) metadata["contigs"] = contigs

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:214, in _prep_bed(bed, max_jitter)
    [212](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:212)     bed = bed.with_columns(strand=pl.lit(1, pl.Int32))
    [213](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:213) else:
--> [214](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:214)     bed = bed.with_columns(
    [215](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:215)         pl.col("strand").replace({"+": 1, "-": -1}, return_dtype=pl.Int32)
    [216](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:216)     )
    [217](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:217) bed = bed.select("chrom", "chromStart", "chromEnd", "strand")
    [218](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:218) with pl.StringCache():

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9806, in DataFrame.with_columns(self, *exprs, **named_exprs)
   [9660](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9660) def with_columns(
   [9661](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9661)     self,
   [9662](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9662)     *exprs: IntoExpr | Iterable[IntoExpr],
   [9663](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9663)     **named_exprs: IntoExpr,
   [9664](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9664) ) -> DataFrame:
   [9665](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9665)     """
   [9666](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9666)     Add columns to this DataFrame.
   [9667](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9667) 
   (...)
   [9804](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9804)     └─────┴──────┴─────────────┘
   [9805](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9805)     """
-> [9806](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/dataframe/frame.py:9806)     return self.lazy().with_columns(*exprs, **named_exprs).collect(_eager=True)

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/_utils/deprecation.py:93, in deprecate_streaming_parameter.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
     [89](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/_utils/deprecation.py:89)         kwargs["engine"] = "in-memory"
     [91](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/_utils/deprecation.py:91)     del kwargs["streaming"]
---> [93](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/_utils/deprecation.py:93) return function(*args, **kwargs)

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/lazyframe/frame.py:2206, in LazyFrame.collect(self, type_coercion, _type_check, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, cluster_with_columns, collapse_joins, no_optimization, engine, background, _check_order, _eager, **_kwargs)
   [2204](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/lazyframe/frame.py:2204) # Only for testing purposes
   [2205](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/lazyframe/frame.py:2205) callback = _kwargs.get("post_opt_callback", callback)
-> [2206](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/polars/lazyframe/frame.py:2206) return wrap_df(ldf.collect(engine, callback))

ComputeError: cannot cast numeric types to 'Categorical'

Conda env

Details

# packages in environment at [/grid/koo/home/schilder/.conda/envs/genome-loader](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/.conda/envs/genome-loader):
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
aiohappyeyeballs          2.4.4              pyhd8ed1ab_1    conda-forge
aiohttp                   3.10.5          py312h41a817b_0    conda-forge
aiohttp-cors              0.7.0              pyhd8ed1ab_1    conda-forge
aiosignal                 1.3.2              pyhd8ed1ab_0    conda-forge
annotated-types           0.7.0              pyhd8ed1ab_1    conda-forge
anyio                     4.8.0              pyhd8ed1ab_0    conda-forge
aom                       3.6.1                h59595ed_0    conda-forge
appdirs                   1.4.4              pyhd3eb1b0_0  
argcomplete               3.6.2                    pypi_0    pypi
argh                      0.31.3                   pypi_0    pypi
argon2-cffi               23.1.0             pyhd8ed1ab_1    conda-forge
argon2-cffi-bindings      21.2.0          py312h66e93f0_5    conda-forge
arrow                     1.3.0              pyhd8ed1ab_1    conda-forge
asciitree                 0.3.3                    pypi_0    pypi
astroid                   3.2.4           py312h06a4308_0  
asttokens                 3.0.0              pyhd8ed1ab_1    conda-forge
async-lru                 2.0.4              pyhd8ed1ab_1    conda-forge
async-timeout             5.0.1              pyhd8ed1ab_1    conda-forge
attrs                     25.3.0                   pypi_0    pypi
awkward                   2.8.1                    pypi_0    pypi
awkward-cpp               45                       pypi_0    pypi
aws-c-auth                0.8.0                h56a2c13_4    conda-forge
aws-c-cal                 0.8.0                hd3f4568_0    conda-forge
aws-c-common              0.9.31               hb9d3cd8_0    conda-forge
aws-c-compression         0.3.0                hf20e7d7_0    conda-forge
aws-c-event-stream        0.5.0                h68c3b0c_2    conda-forge
aws-c-http                0.9.0                hfad4ed3_3    conda-forge
aws-c-io                  0.15.0               h17eb868_2    conda-forge
aws-c-mqtt                0.11.0               h407ecb8_2    conda-forge
aws-c-s3                  0.7.0                hadeddc1_5    conda-forge
aws-c-sdkutils            0.2.0                hf20e7d7_0    conda-forge
aws-checksums             0.2.0                hf20e7d7_0    conda-forge
aws-crt-cpp               0.29.0               h73f0fd4_6    conda-forge
aws-sdk-cpp               1.11.407             h6a6dca0_6    conda-forge
azure-core-cpp            1.14.0               h5cfcd09_0    conda-forge
azure-identity-cpp        1.10.0               h113e628_0    conda-forge
azure-storage-blobs-cpp   12.13.0              h3cf044e_1    conda-forge
azure-storage-common-cpp  12.8.0               h736e048_1    conda-forge
azure-storage-files-datalake-cpp 12.12.0              ha633028_1    conda-forge
babel                     2.17.0             pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.13.3             pyha770c72_0    conda-forge
biopython                 1.78            py312h5eee18b_0  
blas                      1.0                         mkl  
bleach                    6.2.0              pyh29332c3_4    conda-forge
bleach-with-css           6.2.0                h82add2a_4    conda-forge
bokeh                     3.6.2              pyhd8ed1ab_1    conda-forge
brotli                    1.1.0                hb9d3cd8_2    conda-forge
brotli-bin                1.1.0                hb9d3cd8_2    conda-forge
brotli-python             1.1.0           py312h2ec8cdc_2    conda-forge
bzip2                     1.0.8                h4bc722e_7    conda-forge
c-ares                    1.34.4               hb9d3cd8_0    conda-forge
ca-certificates           2025.1.31            hbcca054_0    conda-forge
cachecontrol              0.14.1             pyha770c72_1    conda-forge
cachecontrol-with-filecache 0.14.1             pyhd8ed1ab_1    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.5.0              pyhd8ed1ab_1    conda-forge
certifi                   2025.1.31          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py312h06ac9bb_0    conda-forge
charset-normalizer        3.4.1                    pypi_0    pypi
cleo                      2.2.1              pyhd8ed1ab_0    conda-forge
click                     8.1.8                    pypi_0    pypi
cloudpickle               3.1.0              pyhd8ed1ab_1    conda-forge
colorama                  0.4.6              pyhd8ed1ab_1    conda-forge
coloredlogs               15.0.1                   pypi_0    pypi
colorful                  0.5.6              pyhd8ed1ab_0    conda-forge
comm                      0.2.2              pyhd8ed1ab_1    conda-forge
contourpy                 1.3.1           py312h68727a3_0    conda-forge
crashtest                 0.4.1              pyhd8ed1ab_1    conda-forge
cryptography              44.0.0          py312hda17c39_0    conda-forge
cslug                     1.0.0                    pypi_0    pypi
cuda-cudart               12.4.127                      0    nvidia
cuda-cupti                12.4.127                      0    nvidia
cuda-libraries            12.4.1                        0    nvidia
cuda-nvrtc                12.4.127                      0    nvidia
cuda-nvtx                 12.4.127                      0    nvidia
cuda-opencl               12.6.77                       0    nvidia
cuda-runtime              12.4.1                        0    nvidia
cuda-version              12.6                          3    nvidia
cycler                    0.12.1             pyhd8ed1ab_1    conda-forge
cytoolz                   1.0.1           py312h66e93f0_0    conda-forge
cyvcf2                    0.31.1                   pypi_0    pypi
dask                      2024.12.0          pyhd8ed1ab_1    conda-forge
dask-core                 2024.12.0          pyhd8ed1ab_1    conda-forge
dask-expr                 1.1.20             pyhd8ed1ab_1    conda-forge
dask-glm                  0.3.2                    pypi_0    pypi
dask-ml                   2024.4.4                 pypi_0    pypi
datacache                 1.1.5                      py_0    conda-forge
dbus                      1.13.6               h5008d03_3    conda-forge
debugpy                   1.8.11          py312h2ec8cdc_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_1    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
dill                      0.3.8           py312h06a4308_0  
distlib                   0.3.9              pyhd8ed1ab_1    conda-forge
distributed               2024.12.0          pyhd8ed1ab_1    conda-forge
dulwich                   0.21.7          py312h66e93f0_1    conda-forge
einops                    0.8.1                    pypi_0    pypi
elfutils                  0.192                h7f4e02f_1    conda-forge
ensembl-rest              0.3.4                    pypi_0    pypi
entrypoints               0.4                pyhd8ed1ab_1    conda-forge
exceptiongroup            1.2.2              pyhd8ed1ab_1    conda-forge
executing                 2.1.0              pyhd8ed1ab_1    conda-forge
expat                     2.6.4                h5888daf_0    conda-forge
fasteners                 0.19                     pypi_0    pypi
ffmpeg                    4.4.2           gpl_hdf48244_113    conda-forge
filelock                  3.16.1             pyhd8ed1ab_1    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hd3eb1b0_0  
font-ttf-inconsolata      2.001                hcb22688_0  
font-ttf-source-code-pro  2.030                hd3eb1b0_0  
font-ttf-ubuntu           0.83                 h8b1ccd4_0  
fontconfig                2.15.0               h7e30c49_1    conda-forge
fonts-anaconda            1                    h8fa9717_0  
fonts-conda-ecosystem     1                    hd3eb1b0_0  
fonttools                 4.56.0          py312h178313f_0    conda-forge
fqdn                      1.5.1              pyhd8ed1ab_1    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
frozenlist                1.5.0           py312h66e93f0_0    conda-forge
fsspec                    2025.3.2                 pypi_0    pypi
genvarloader              0.10.5                   pypi_0    pypi
gettext                   0.22.5               he02047a_3    conda-forge
gettext-tools             0.22.5               he02047a_3    conda-forge
gffutils                  0.13                     pypi_0    pypi
gflags                    2.2.2             h5888daf_1005    conda-forge
giflib                    5.2.2                h5eee18b_0  
glog                      0.7.1                hbabe93e_0    conda-forge
gmp                       6.2.1                h295c915_3  
gnutls                    3.7.9                hb077bed_0    conda-forge
google-api-core           2.24.0             pyhd8ed1ab_0    conda-forge
google-auth               2.37.0             pyhd8ed1ab_0    conda-forge
googleapis-common-protos  1.66.0             pyhff2d567_0    conda-forge
grpcio                    1.65.5          py312h374181b_0    conda-forge
gtfparse                  2.5.0                    pypi_0    pypi
h11                       0.14.0             pyhd8ed1ab_1    conda-forge
h2                        4.1.0              pyhd8ed1ab_1    conda-forge
haptools                  0.5.0              pyhdfd78af_0    bioconda
hirola                    0.3.0                    pypi_0    pypi
hpack                     4.0.0              pyhd8ed1ab_1    conda-forge
htslib                    1.21                 h566b1c6_1    bioconda
httpcore                  1.0.7              pyh29332c3_1    conda-forge
httpx                     0.28.1             pyhd8ed1ab_0    conda-forge
humanfriendly             10.0                     pypi_0    pypi
hyperframe                6.0.1              pyhd8ed1ab_1    conda-forge
idna                      3.10               pyhd8ed1ab_1    conda-forge
importlib-metadata        8.5.0              pyha770c72_1    conda-forge
importlib_resources       6.4.5              pyhd8ed1ab_1    conda-forge
intel-openmp              2022.0.1          h06a4308_3633  
intervaltree              3.1.0                    pypi_0    pypi
iprogress                 0.4                      pypi_0    pypi
ipykernel                 6.29.5             pyh3099207_0    conda-forge
ipython                   8.30.0             pyh707e725_0    conda-forge
isoduration               20.11.0            pyhd8ed1ab_1    conda-forge
isort                     5.13.2          py312h06a4308_0  
jaraco.classes            3.4.0              pyhd8ed1ab_2    conda-forge
jedi                      0.19.2             pyhd8ed1ab_1    conda-forge
jeepney                   0.8.0              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_1    conda-forge
joblib                    1.4.2              pyhd8ed1ab_1    conda-forge
json5                     0.10.0             pyhd8ed1ab_1    conda-forge
jsonpointer               3.0.0           py312h7900ff3_1    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_1    conda-forge
jsonschema-specifications 2024.10.1          pyhd8ed1ab_1    conda-forge
jsonschema-with-format-nongpl 4.23.0               hd8ed1ab_1    conda-forge
jupyter-client            8.6.3                    pypi_0    pypi
jupyter-lsp               2.2.5              pyhd8ed1ab_1    conda-forge
jupyter_client            7.4.9              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2              pyh31011fe_1    conda-forge
jupyter_events            0.12.0             pyh29332c3_0    conda-forge
jupyter_server            2.15.0             pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.5.3              pyhd8ed1ab_1    conda-forge
jupyterlab                4.3.5              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.3.0              pyhd8ed1ab_2    conda-forge
jupyterlab_server         2.27.3             pyhd8ed1ab_1    conda-forge
keyring                   24.3.1             pyha804496_1    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.8           py312h84d6215_0    conda-forge
krb5                      1.21.3               h659f571_0    conda-forge
lame                      3.100                h7b6447c_0  
lcms2                     2.16                 hb7c19ff_0    conda-forge
ld_impl_linux-64          2.43                 h712a8e2_2    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20240722.0      cxx17_h5888daf_1    conda-forge
libarchive                3.7.7                hadbb8c3_0    conda-forge
libarrow                  18.0.0           ha5db6c2_0_cpu    conda-forge
libarrow-acero            18.0.0           h5888daf_0_cpu    conda-forge
libarrow-dataset          18.0.0           h5888daf_0_cpu    conda-forge
libarrow-substrait        18.0.0           he882d9a_0_cpu    conda-forge
libasprintf               0.22.5               he8f35ee_3    conda-forge
libasprintf-devel         0.22.5               he8f35ee_3    conda-forge
libblas                   3.9.0            16_linux64_mkl    conda-forge
libbrotlicommon           1.1.0                hb9d3cd8_2    conda-forge
libbrotlidec              1.1.0                hb9d3cd8_2    conda-forge
libbrotlienc              1.1.0                hb9d3cd8_2    conda-forge
libcblas                  3.9.0            16_linux64_mkl    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcublas                 12.4.5.8                      0    nvidia
libcufft                  11.2.1.3                      0    nvidia
libcufile                 1.11.1.6                      0    nvidia
libcurand                 10.3.7.77                     0    nvidia
libcurl                   8.11.1               h332b0f4_0    conda-forge
libcusolver               11.6.1.9                      0    nvidia
libcusparse               12.3.1.170                    0    nvidia
libdeflate                1.22                 hb9d3cd8_0    conda-forge
libdrm                    2.4.124              hb9d3cd8_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libegl                    1.7.0                ha4b6fd6_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libevent                  2.1.12               hf998b51_1    conda-forge
libexpat                  2.6.4                h5888daf_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc                    14.2.0               h77fa898_1    conda-forge
libgcc-ng                 14.2.0               h69a702a_1    conda-forge
libgettextpo              0.22.5               he02047a_3    conda-forge
libgettextpo-devel        0.22.5               he02047a_3    conda-forge
libgfortran               14.2.0               h69a702a_1    conda-forge
libgfortran5              14.2.0               hd5240d6_1    conda-forge
libgl                     1.7.0                ha4b6fd6_2    conda-forge
libglib                   2.82.2               h2ff4ddf_0    conda-forge
libglvnd                  1.7.0                ha4b6fd6_2    conda-forge
libglx                    1.7.0                ha4b6fd6_2    conda-forge
libgomp                   14.2.0               h77fa898_1    conda-forge
libgoogle-cloud           2.30.0               h438788a_0    conda-forge
libgoogle-cloud-storage   2.30.0               h0121fbd_0    conda-forge
libgrpc                   1.65.5               hf5c653b_0    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libidn2                   2.3.7                hd590300_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0            16_linux64_mkl    conda-forge
libllvm14                 14.0.6               hcd5def8_4    conda-forge
liblzma                   5.6.3                hb9d3cd8_1    conda-forge
liblzma-devel             5.6.3                hb9d3cd8_1    conda-forge
libmicrohttpd             1.0.1                h97afed2_0    conda-forge
libnghttp2                1.64.0               h161d5f1_0    conda-forge
libnpp                    12.2.5.30                     0    nvidia
libnsl                    2.0.1                hd590300_0    conda-forge
libnvfatbin               12.6.77                       0    nvidia
libnvjitlink              12.4.127                      0    nvidia
libnvjpeg                 12.3.1.117                    0    nvidia
libopenblas               0.3.28          pthreads_h94d23a6_1    conda-forge
libparquet                18.0.0           h6bd9018_0_cpu    conda-forge
libpciaccess              0.18                 hd590300_0    conda-forge
libpng                    1.6.44               hadc24fc_0    conda-forge
libprotobuf               5.27.5               h5b01275_2    conda-forge
libre2-11                 2024.07.02           hbbce691_1    conda-forge
libsodium                 1.0.20               h4ab18f5_0    conda-forge
libsqlite                 3.47.2               hee588c1_0    conda-forge
libssh2                   1.11.1               hf672d98_0    conda-forge
libstdcxx                 14.2.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              14.2.0               h4852527_1    conda-forge
libtasn1                  4.19.0               h166bdaf_0    conda-forge
libthrift                 0.21.0               h0e7cc3e_0    conda-forge
libtiff                   4.7.0                hc4654cb_2    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libunwind                 1.6.2                h9c3ff4c_0    conda-forge
libutf8proc               2.8.0                hf23e847_1    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libva                     2.22.0               h8a09558_1    conda-forge
libvpx                    1.13.1               h6a678d5_0  
libwebp                   1.4.0                h2c329e2_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxcb                    1.17.0               h8a09558_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.13.5               h0d44e9d_1    conda-forge
libzlib                   1.3.1                hb9d3cd8_2    conda-forge
linkify-it-py             2.0.3              pyhd8ed1ab_1    conda-forge
llvm-openmp               15.0.7               h0cdce71_0    conda-forge
llvmlite                  0.44.0                   pypi_0    pypi
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
loguru                    0.7.3                    pypi_0    pypi
lz4                       4.3.3           py312hb3f7f12_1    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              hd590300_1001    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_1    conda-forge
markupsafe                3.0.2           py312h178313f_1    conda-forge
matplotlib-base           3.10.1          py312hd3ec401_0    conda-forge
matplotlib-inline         0.1.7              pyhd8ed1ab_1    conda-forge
mccabe                    0.7.0              pyhd3eb1b0_0  
mdit-py-plugins           0.4.2              pyhd8ed1ab_1    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_1    conda-forge
memoized-property         1.0.3              pyhd8ed1ab_1    conda-forge
memray                    1.15.0          py312h9f2de7a_0    conda-forge
mistune                   3.1.2              pyhd8ed1ab_0    conda-forge
mkl                       2022.1.0           hc2b9512_224  
ml-dtypes                 0.5.0                    pypi_0    pypi
mock                      4.0.3              pyhd3eb1b0_0  
more-itertools            10.6.0                   pypi_0    pypi
mpmath                    1.3.0           py312h06a4308_0  
msgpack-python            1.1.0           py312h68727a3_0    conda-forge
multidict                 6.1.0           py312h178313f_2    conda-forge
multimethod               1.9.1              pyhd8ed1ab_0    conda-forge
multipledispatch          1.0.0                    pypi_0    pypi
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
mypy_extensions           1.0.0              pyha770c72_1    conda-forge
natsort                   8.4.0              pyh29332c3_1    conda-forge
nbclient                  0.10.2             pyhd8ed1ab_0    conda-forge
nbconvert-core            7.16.6             pyh29332c3_0    conda-forge
nbformat                  5.10.4             pyhd8ed1ab_1    conda-forge
ncls                      0.0.68                   pypi_0    pypi
ncurses                   6.5                  he02047a_1    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_1    conda-forge
nettle                    3.9.1                h7ab15ed_0    conda-forge
networkx                  3.3             py312h06a4308_0  
notebook                  7.3.2              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.4              pyhd8ed1ab_1    conda-forge
numba                     0.61.2                   pypi_0    pypi
numcodecs                 0.13.1                   pypi_0    pypi
numpy                     2.2.4                    pypi_0    pypi
opencensus                0.11.3             pyhd8ed1ab_0    conda-forge
opencensus-context        0.1.3           py312h7900ff3_3    conda-forge
openh264                  2.3.1                hcb278e6_2    conda-forge
openjpeg                  2.5.3                h5fbd93e_0    conda-forge
openssl                   3.4.1                h7b32b05_0    conda-forge
orc                       2.0.2                h690cf93_1    conda-forge
overrides                 7.7.0              pyhd8ed1ab_1    conda-forge
p11-kit                   0.24.1               hc5aa10d_0    conda-forge
packaging                 24.2               pyhd8ed1ab_2    conda-forge
pandas                    2.2.3           py312hf9745cd_1    conda-forge
pandera                   0.23.1                   pypi_0    pypi
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.4              pyhd8ed1ab_1    conda-forge
partd                     1.4.2              pyhd8ed1ab_0    conda-forge
pcre2                     10.44                hba22ea6_2    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_1    conda-forge
pgenlib                   0.92.0                   pypi_0    pypi
pickleshare               0.7.5           pyhd8ed1ab_1004    conda-forge
pillow                    11.0.0          py312h7b63e92_0    conda-forge
pip                       25.0.1             pyh8b19718_0    conda-forge
pkginfo                   1.12.0             pyhd8ed1ab_1    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_2    conda-forge
platformdirs              4.3.7                    pypi_0    pypi
poetry                    1.8.5              pyha804496_0    conda-forge
poetry-core               1.9.1              pyhd8ed1ab_1    conda-forge
poetry-plugin-export      1.8.0              pyhd8ed1ab_1    conda-forge
polars                    1.27.1                   pypi_0    pypi
pooch                     1.8.2                    pypi_0    pypi
progressbar33             2.4                pyhd8ed1ab_1    conda-forge
prometheus_client         0.21.1             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.48             pyha770c72_1    conda-forge
propcache                 0.2.1           py312h66e93f0_0    conda-forge
proto-plus                1.25.0             pyhd8ed1ab_1    conda-forge
protobuf                  5.27.5          py312h2ec8cdc_0    conda-forge
psutil                    6.1.0           py312h66e93f0_0    conda-forge
pthread-stubs             0.4               hb9d3cd8_1002    conda-forge
ptyprocess                0.7.0              pyhd8ed1ab_1    conda-forge
pure_eval                 0.2.3              pyhd8ed1ab_1    conda-forge
py-spy                    0.4.0                h4c5a871_1    conda-forge
pyarrow                   19.0.1                   pypi_0    pypi
pyasn1                    0.6.1              pyhd8ed1ab_2    conda-forge
pyasn1-modules            0.4.1              pyhd8ed1ab_1    conda-forge
pybigwig                  0.3.24                   pypi_0    pypi
pybind11                  2.13.6                   pypi_0    pypi
pycparser                 2.22               pyh29332c3_1    conda-forge
pydantic                  2.11.3                   pypi_0    pypi
pydantic-core             2.33.1                   pypi_0    pypi
pyensembl                 2.3.13             pyh7cba7a3_0    bioconda
pyfaidx                   0.8.1.3                  pypi_0    pypi
pygments                  2.19.1                   pypi_0    pypi
pylint                    3.2.7           py312h06a4308_0  
pyopenssl                 24.3.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.2.2              pyhd8ed1ab_0    conda-forge
pyproject_hooks           1.2.0              pyhd8ed1ab_1    conda-forge
pyranges                  0.1.4                    pypi_0    pypi
pysam                     0.23.0                   pypi_0    pypi
pysocks                   1.7.1              pyha55dd90_7    conda-forge
python                    3.12.8          h9e4cc4f_1_cpython    conda-forge
python-build              1.2.2.post1        pyhff2d567_1    conda-forge
python-dateutil           2.9.0.post0        pyhff2d567_1    conda-forge
python-fastjsonschema     2.21.1             pyhd8ed1ab_0    conda-forge
python-installer          0.7.0              pyhff2d567_1    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python_abi                3.12                    5_cp312    conda-forge
pytorch                   2.5.1           py3.12_cuda12.4_cudnn9.1.0_0    pytorch
pytorch-cuda              12.4                 hc786d27_7    pytorch
pytorch-mutex             1.0                        cuda    pytorch
pytz                      2025.2                   pypi_0    pypi
pyu2f                     0.1.5              pyhd8ed1ab_1    conda-forge
pyyaml                    6.0.2           py312h66e93f0_1    conda-forge
pyzmq                     24.0.1          py312h5eee18b_0  
qhull                     2020.2               h434a139_5    conda-forge
ray-core                  2.40.0          py312h6630fa3_0    conda-forge
ray-default               2.40.0          py312h6dd12e9_0    conda-forge
re2                       2024.07.02           h77b4e00_1    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_1    conda-forge
requests                  2.32.3             pyhd8ed1ab_1    conda-forge
requests-toolbelt         1.0.0              pyhd8ed1ab_1    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_1    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rich                      14.0.0                   pypi_0    pypi
rpds-py                   0.22.3          py312h12e396e_0    conda-forge
rsa                       4.9                pyhd8ed1ab_1    conda-forge
s2n                       1.5.6                h0e56266_0    conda-forge
scikit-learn              1.6.1                    pypi_0    pypi
scipy                     1.15.2                   pypi_0    pypi
secretstorage             3.3.3           py312h7900ff3_3    conda-forge
send2trash                1.8.3              pyh0d859eb_1    conda-forge
seqpro                    0.2.0                    pypi_0    pypi
serializable              0.2.1              pyhd8ed1ab_0    conda-forge
setproctitle              1.3.4           py312h66e93f0_0    conda-forge
setuptools                75.6.0             pyhff2d567_1    conda-forge
sgkit                     0.9.0                    pypi_0    pypi
shellingham               1.5.4              pyhd8ed1ab_1    conda-forge
simplejson                3.19.2          py312h5eee18b_0  
six                       1.17.0             pyhd8ed1ab_0    conda-forge
smart_open                7.0.5              pyhd8ed1ab_1    conda-forge
snappy                    1.2.1                h8bd8927_1    conda-forge
sniffio                   1.3.1              pyhd8ed1ab_1    conda-forge
sorted-nearest            0.0.39                   pypi_0    pypi
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
sparse                    0.15.5                   pypi_0    pypi
stack_data                0.6.3              pyhd8ed1ab_1    conda-forge
svt-av1                   1.4.1                hcb278e6_0    conda-forge
sympy                     1.13.2          py312h06a4308_0  
tabix                     1.11                 hdfd78af_0    bioconda
tabulate                  0.9.0                    pypi_0    pypi
tbb                       2022.1.0                 pypi_0    pypi
tblib                     3.0.0              pyhd8ed1ab_1    conda-forge
tcmlib                    1.3.0                    pypi_0    pypi
tensorstore               0.1.71                   pypi_0    pypi
terminado                 0.18.1             pyh0d859eb_0    conda-forge
textual                   1.0.0              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.6.0                    pypi_0    pypi
tinycss2                  1.4.0              pyhd8ed1ab_0    conda-forge
tinytimer                 0.0.0                      py_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomli                     2.2.1              pyhd8ed1ab_1    conda-forge
tomlkit                   0.13.2             pyha770c72_1    conda-forge
toolz                     1.0.0              pyhd8ed1ab_1    conda-forge
torchaudio                2.5.1               py312_cu124    pytorch
torchtriton               3.1.0                     py312    pytorch
torchvision               0.20.1              py312_cu124    pytorch
tornado                   6.4.2           py312h66e93f0_0    conda-forge
tqdm                      4.67.1                   pypi_0    pypi
traitlets                 5.14.3             pyhd8ed1ab_1    conda-forge
trove-classifiers         2024.10.21.16      pyhd8ed1ab_1    conda-forge
typechecks                0.1.0              pyhd8ed1ab_1    conda-forge
typeguard                 4.4.2                    pypi_0    pypi
typer                     0.15.2                   pypi_0    pypi
types-python-dateutil     2.9.0.20241206     pyhd8ed1ab_0    conda-forge
typing-extensions         4.13.2                   pypi_0    pypi
typing-inspection         0.4.0                    pypi_0    pypi
typing_inspect            0.9.0              pyhd8ed1ab_1    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_1    conda-forge
tzdata                    2025.2                   pypi_0    pypi
uc-micro-py               1.0.3              pyhd8ed1ab_1    conda-forge
unicodedata2              16.0.0          py312h66e93f0_0    conda-forge
uri-template              1.3.0              pyhd8ed1ab_1    conda-forge
urllib3                   2.4.0                    pypi_0    pypi
virtualenv                20.28.0            pyhd8ed1ab_0    conda-forge
wayland                   1.23.1               h3e06ad9_0    conda-forge
wayland-protocols         1.37                 hd8ed1ab_0    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_1    conda-forge
webcolors                 24.11.1            pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_3    conda-forge
websocket-client          1.8.0              pyhd8ed1ab_1    conda-forge
wheel                     0.45.1             pyhd8ed1ab_1    conda-forge
wrapt                     1.17.0          py312h66e93f0_0    conda-forge
x264                      1!164.3095           h166bdaf_2    conda-forge
x265                      3.5                  h924138e_3    conda-forge
xarray                    2024.11.0          pyhd8ed1ab_0    conda-forge
xorg-libx11               1.8.10               h4f16b4b_1    conda-forge
xorg-libxau               1.0.12               hb9d3cd8_0    conda-forge
xorg-libxdmcp             1.1.5                hb9d3cd8_0    conda-forge
xorg-libxext              1.3.6                hb9d3cd8_0    conda-forge
xorg-libxfixes            6.0.1                hb9d3cd8_0    conda-forge
xyzservices               2024.9.0           pyhd8ed1ab_1    conda-forge
xz                        5.6.3                hbcc6ac9_1    conda-forge
xz-gpl-tools              5.6.3                hbcc6ac9_1    conda-forge
xz-tools                  5.6.3                hb9d3cd8_1    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yarl                      1.18.3          py312h66e93f0_0    conda-forge
zarr                      2.18.4                   pypi_0    pypi
zeromq                    4.3.5                h3b0a872_7    conda-forge
zict                      3.0.0              pyhd8ed1ab_1    conda-forge
zipp                      3.21.0             pyhd8ed1ab_1    conda-forge
zlib                      1.3.1                hb9d3cd8_2    conda-forge
zstandard                 0.23.0          py312hef9b889_1    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge

[bschilder]

Changing the "strand" col from Categorical to String get paste the error.

Though it might be good to handle these scenarios automatically within GVL.

chr22_genes = (
    gtf
    .filter(pl.col("seqname") == "22")
    .filter(pl.col("feature") == "exon")
    .rename({"seqname": "chrom", "start": "chromStart", "end": "chromEnd"})
    .with_columns(pl.col("strand").cast(pl.Utf8))
)
chr22_genes.head()

Though now a new (I think unrelated error) comes up:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:326, in Records.read_vcf(vcf_path)
    [325](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:325) try:
--> [326](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:326)     n_variants = cast(int, vcf.num_records)
    [327](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:327) except ValueError as no_idx_err:

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/cyvcf2/cyvcf2.pyx:724, in cyvcf2.cyvcf2.VCF.num_records.__get__()

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/cyvcf2/cyvcf2.pyx:700, in cyvcf2.cyvcf2.VCF._num_records()

ValueError: File must be indexed to compute num_records (tip: use bcftools index)

During handling of the above exception, another exception occurred:

FileNotFoundError                         Traceback (most recent call last)
Cell In[35], [line 1](vscode-notebook-cell:?execution_count=35&line=1)
----> [1](vscode-notebook-cell:?execution_count=35&line=1) gvl.write(
      [2](vscode-notebook-cell:?execution_count=35&line=2)     path="1kg_chr22.gvl",
      [3](vscode-notebook-cell:?execution_count=35&line=3)     bed=chr22_genes,
      [4](vscode-notebook-cell:?execution_count=35&line=4)     variants=vcf_path,
      [5](vscode-notebook-cell:?execution_count=35&line=5)     max_jitter=0,
      [6](vscode-notebook-cell:?execution_count=35&line=6)     overwrite=True
      [7](vscode-notebook-cell:?execution_count=35&line=7) )

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:124, in write(path, bed, variants, bigwigs, samples, max_jitter, overwrite, max_mem, phased, ccf_field)
    [119](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:119)     raise ValueError(
    [120](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:120)         f"Phased argument ({phased}) does not match phased status of Variants object ({variants.phased})."
    [121](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:121)     )
    [123](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:123) if isinstance(variants, (str, Path)):
--> [124](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:124)     variants = Variants.from_file(variants, phased, ccf_field)
    [126](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:126) if unavailable_contigs := set(contigs) - {
    [127](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:127)     _normalize_contig_name(c, contigs) for c in variants.records.contigs
    [128](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:128) }:
    [129](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:129)     logger.warning(
    [130](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:130)         f"Contigs in queries {unavailable_contigs} are not found in the VCF."
    [131](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:131)     )

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:78, in Variants.from_file(cls, path, phased, ccf_field)
     [75](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:75)     first_path = next(iter(path.values()))
     [77](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:77) if path_is_vcf(first_path):
---> [78](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:78)     return cls._from_vcf(path, phased, ccf_field)
     [79](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:79) elif path_is_pgen(first_path):
     [80](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:80)     return cls._from_pgen(path, phased)

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:147, in Variants._from_vcf(cls, vcf, phased, ccf_field)
    [142](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:142)     if ccf_field_info["Number"] not in {"1", "A"}:
    [143](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:143)         raise CCFFieldError(
    [144](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:144)             f"CCF field '{ccf_field}' must have Number equal to '1' or 'A'."
    [145](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:145)         )
--> [147](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:147) records = Records.from_vcf(vcf)
    [149](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:149) genotypes = VCFGenos(vcf, records.contig_offsets)
    [150](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:150) return cls(records, genotypes, phased, ccf_field)

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:283, in Records.from_vcf(cls, vcf)
    [280](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:280)     return cls.from_gvl_arrow(arrow_paths)
    [282](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:282) if multi_contig_source:
--> [283](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:283)     start_df = cls.read_vcf(vcf["_all"])
    [284](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:284)     start_dfs = {
    [285](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:285)         cast(str, c[0]): v
    [286](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:286)         for c, v in start_df.partition_by(
    [287](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:287)             "#CHROM", as_dict=True, maintain_order=True
    [288](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:288)         ).items()
    [289](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:289)     }
    [290](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:290)     del start_df

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:330, in Records.read_vcf(vcf_path)
    [328](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:328) logger.info(f"VCF file {vcf_path} appears to be unindexed, indexing...")
    [329](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:329) try:
--> [330](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:330)     subprocess.run(
    [331](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:331)         ["bcftools", "index", vcf_path], check=True, capture_output=True
    [332](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:332)     )
    [333](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:333) except subprocess.CalledProcessError as bcftools_err:
    [334](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:334)     logger.error(f"Failed to index VCF file {vcf_path}.")

File ~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:548, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    [545](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:545)     kwargs['stdout'] = PIPE
    [546](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:546)     kwargs['stderr'] = PIPE
--> [548](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:548) with Popen(*popenargs, **kwargs) as process:
    [549](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:549)     try:
    [550](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:550)         stdout, stderr = process.communicate(input, timeout=timeout)

File ~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1026, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)
   [1022](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1022)         if self.text_mode:
   [1023](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1023)             self.stderr = io.TextIOWrapper(self.stderr,
   [1024](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1024)                     encoding=encoding, errors=errors)
-> [1026](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1026)     self._execute_child(args, executable, preexec_fn, close_fds,
   [1027](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1027)                         pass_fds, cwd, env,
   [1028](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1028)                         startupinfo, creationflags, shell,
   [1029](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1029)                         p2cread, p2cwrite,
   [1030](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1030)                         c2pread, c2pwrite,
   [1031](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1031)                         errread, errwrite,
   [1032](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1032)                         restore_signals,
   [1033](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1033)                         gid, gids, uid, umask,
   [1034](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1034)                         start_new_session, process_group)
   [1035](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1035) except:
   [1036](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1036)     # Cleanup if the child failed starting.
   [1037](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1037)     for f in filter(None, (self.stdin, self.stdout, self.stderr)):

File ~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1955, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session, process_group)
   [1953](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1953)     err_msg = os.strerror(errno_num)
   [1954](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1954) if err_filename is not None:
-> [1955](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1955)     raise child_exception_type(errno_num, err_msg, err_filename)
   [1956](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1956) else:
   [1957](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:1957)     raise child_exception_type(errno_num, err_msg)

FileNotFoundError: [Errno 2] No such file or directory: 'bcftools'

Is bcftools a hard requirement for GVL?

[bschilder]

Installing bcftools into the conda env that my Jupyter notebook is connected gets rid of the FileNotFoundError: [Errno 2] No such file or directory: 'bcftools' , but a new one appears:

conda install bioconda::bcftools

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:326, in Records.read_vcf(vcf_path)
    [325](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:325) try:
--> [326](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:326)     n_variants = cast(int, vcf.num_records)
    [327](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:327) except ValueError as no_idx_err:

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/cyvcf2/cyvcf2.pyx:724, in cyvcf2.cyvcf2.VCF.num_records.__get__()

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/cyvcf2/cyvcf2.pyx:700, in cyvcf2.cyvcf2.VCF._num_records()

ValueError: File must be indexed to compute num_records (tip: use bcftools index)

The above exception was the direct cause of the following exception:

CalledProcessError                        Traceback (most recent call last)
Cell In[7], [line 1](vscode-notebook-cell:?execution_count=7&line=1)
----> [1](vscode-notebook-cell:?execution_count=7&line=1) gvl.write(
      [2](vscode-notebook-cell:?execution_count=7&line=2)     path="1kg_chr22.gvl",
      [3](vscode-notebook-cell:?execution_count=7&line=3)     bed=chr22_genes,
      [4](vscode-notebook-cell:?execution_count=7&line=4)     variants=vcf_path,
      [5](vscode-notebook-cell:?execution_count=7&line=5)     max_jitter=0,
      [6](vscode-notebook-cell:?execution_count=7&line=6)     overwrite=True
      [7](vscode-notebook-cell:?execution_count=7&line=7) )

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:124, in write(path, bed, variants, bigwigs, samples, max_jitter, overwrite, max_mem, phased, ccf_field)
    [119](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:119)     raise ValueError(
    [120](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:120)         f"Phased argument ({phased}) does not match phased status of Variants object ({variants.phased})."
    [121](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:121)     )
    [123](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:123) if isinstance(variants, (str, Path)):
--> [124](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:124)     variants = Variants.from_file(variants, phased, ccf_field)
    [126](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:126) if unavailable_contigs := set(contigs) - {
    [127](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:127)     _normalize_contig_name(c, contigs) for c in variants.records.contigs
    [128](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:128) }:
    [129](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:129)     logger.warning(
    [130](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:130)         f"Contigs in queries {unavailable_contigs} are not found in the VCF."
    [131](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:131)     )

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:78, in Variants.from_file(cls, path, phased, ccf_field)
     [75](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:75)     first_path = next(iter(path.values()))
     [77](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:77) if path_is_vcf(first_path):
---> [78](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:78)     return cls._from_vcf(path, phased, ccf_field)
     [79](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:79) elif path_is_pgen(first_path):
     [80](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:80)     return cls._from_pgen(path, phased)

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:147, in Variants._from_vcf(cls, vcf, phased, ccf_field)
    [142](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:142)     if ccf_field_info["Number"] not in {"1", "A"}:
    [143](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:143)         raise CCFFieldError(
    [144](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:144)             f"CCF field '{ccf_field}' must have Number equal to '1' or 'A'."
    [145](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:145)         )
--> [147](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:147) records = Records.from_vcf(vcf)
    [149](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:149) genotypes = VCFGenos(vcf, records.contig_offsets)
    [150](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:150) return cls(records, genotypes, phased, ccf_field)

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:283, in Records.from_vcf(cls, vcf)
    [280](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:280)     return cls.from_gvl_arrow(arrow_paths)
    [282](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:282) if multi_contig_source:
--> [283](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:283)     start_df = cls.read_vcf(vcf["_all"])
    [284](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:284)     start_dfs = {
    [285](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:285)         cast(str, c[0]): v
    [286](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:286)         for c, v in start_df.partition_by(
    [287](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:287)             "#CHROM", as_dict=True, maintain_order=True
    [288](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:288)         ).items()
    [289](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:289)     }
    [290](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:290)     del start_df

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:337, in Records.read_vcf(vcf_path)
    [335](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:335)     logger.error(f"stdout:\n{bcftools_err.stdout.decode()}")
    [336](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:336)     logger.error(f"stderr:\n{bcftools_err.stderr.decode()}")
--> [337](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:337)     raise bcftools_err from no_idx_err
    [338](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:338) vcf = cyvcf2.VCF(str(vcf_path))  # pyright: ignore
    [339](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:339) n_variants = cast(int, vcf.num_records)

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:330, in Records.read_vcf(vcf_path)
    [328](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:328) logger.info(f"VCF file {vcf_path} appears to be unindexed, indexing...")
    [329](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:329) try:
--> [330](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:330)     subprocess.run(
    [331](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:331)         ["bcftools", "index", vcf_path], check=True, capture_output=True
    [332](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:332)     )
    [333](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:333) except subprocess.CalledProcessError as bcftools_err:
    [334](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_records.py:334)     logger.error(f"Failed to index VCF file {vcf_path}.")

File ~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:571, in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    [569](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:569)     retcode = process.poll()
    [570](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:570)     if check and retcode:
--> [571](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:571)         raise CalledProcessError(retcode, process.args,
    [572](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:572)                                  output=stdout, stderr=stderr)
    [573](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/subprocess.py:573) return CompletedProcess(process.args, retcode, stdout, stderr)

CalledProcessError: Command '['bcftools', 'index', PosixPath('/grid/koo/home/schilder/.cache/pooch/1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.vcf.gz')]' returned non-zero exit status 127.

[bschilder]

Downloading the tbi file for the VCF in the same location does, however, bypass the bcftools issue.

pooch.retrieve(
    url="https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working/20220422_3202_phased_SNV_INDEL_SV/1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.vcf.gz.tbi",
    known_hash="016cbb66398337629e4b06de238703a17bbfbc25f48f7f660876e8968e2da4de",
    progressbar=True,
    fname="1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.vcf.gz.tbi",
)

Not sure why the bcftools index command isn't working though.

Here is the version I'm using

(genome-loader) [schilder@bamdev2 ProHap]$ conda list bcftools
# packages in environment at /grid/koo/home/schilder/.conda/envs/genome-loader:
#
# Name                    Version                   Build  Channel
bcftools                  1.21                 h3a4d415_1    bioconda

Think this might be due to conda not installing bcftools properly though:

(genome-loader) [schilder@bamdev2 ProHap]$ bcftools
bcftools: error while loading shared libraries: libgsl.so.25: cannot open shared object file: No such file or directory

Related issue:

• bcftools: error while loading shared libraries: libgsl.so.25: cannot open shared object file: No such file or directory samtools/bcftools#1698

[bschilder]

Ah actually, even after bypassing the bcftools issue, this error came up midway through processing :

gvl.write(
    path="1kg_chr22.gvl",
    bed=chr22_genes,
    variants=vcf_path,
    max_jitter=0,
    overwrite=True
)

Scanning variants from 1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.vcf.gz: 100%|██████████| 1066557/1066557 [02:37<00:00, 6762.44it/s]
Reading genotypes for 4363 regions on chromosome 22:   8%|▊         | 1/13 [02:43<32:42, 163.57s/it]

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[10], [line 1](vscode-notebook-cell:?execution_count=10&line=1)
----> [1](vscode-notebook-cell:?execution_count=10&line=1) gvl.write(
      [2](vscode-notebook-cell:?execution_count=10&line=2)     path="1kg_chr22.gvl",
      [3](vscode-notebook-cell:?execution_count=10&line=3)     bed=chr22_genes,
      [4](vscode-notebook-cell:?execution_count=10&line=4)     variants=vcf_path,
      [5](vscode-notebook-cell:?execution_count=10&line=5)     max_jitter=0,
      [6](vscode-notebook-cell:?execution_count=10&line=6)     overwrite=True
      [7](vscode-notebook-cell:?execution_count=10&line=7) )

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:174, in write(path, bed, variants, bigwigs, samples, max_jitter, overwrite, max_mem, phased, ccf_field)
    [172](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:172) if variants is not None:
    [173](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:173)     logger.info("Writing genotypes.")
--> [174](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:174)     gvl_bed = _write_variants(
    [175](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:175)         path,
    [176](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:176)         gvl_bed,
    [177](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:177)         variants,
    [178](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:178)         samples,
    [179](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:179)         max_mem,
    [180](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:180)     )
    [181](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:181)     if isinstance(variants.genotypes, VCFGenos):
    [182](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:182)         variants.genotypes.close()

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:367, in _write_variants(path, bed, variants, samples, max_mem)
    [361](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:361) pbar.set_description(
    [362](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:362)     f"Reading genotypes for {n_regions} regions on chromosome {contig}"
    [363](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:363) )
    [365](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:365) _contig = _normalize_contig_name(contig, variants.records.contigs)
--> [367](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:367) genos, chunk_max_ends = _read_variants_chunk(
    [368](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:368)     _contig,
    [369](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:369)     starts,
    [370](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:370)     ends,
    [371](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:371)     rel_s_idxs,
    [372](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:372)     rel_e_idxs,
    [373](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:373)     variants,
    [374](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:374)     _samples,
    [375](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:375)     sample_idx,
    [376](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:376) )
    [378](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:378) max_ends[last_max_end_idx : last_max_end_idx + n_regions] = (
    [379](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:379)     chunk_max_ends
    [380](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:380) )
    [381](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:381) last_max_end_idx += n_regions

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:464, in _read_variants_chunk(contig, starts, ends, rel_s_idxs, rel_e_idxs, variants, samples, sample_idxs)
    [460](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:460) logger.debug("read genotypes")
    [461](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:461) if variants.phased:
    [462](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:462)     genos = cast(
    [463](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:463)         DenseGenotypes | None,
--> [464](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:464)         variants._read(contig, starts, ends, sample=samples),
    [465](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:465)     )
    [466](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:466) else:
    [467](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_dataset/_write.py:467)     assert variants.ccf_field is not None

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:296, in Variants._read(self, contig, starts, ends, sample, ploid)
    [294](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:294) else:
    [295](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:295)     if self.ccf_field is None:
--> [296](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:296)         genos = self.genotypes.read(
    [297](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:297)             _contig, starts, ends, sample_idxs, ploid, n_variants
    [298](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:298)         )
    [299](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:299)     else:
    [300](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:300)         genos, ccfs = self.genotypes.read_genos_and_ccfs(
    [301](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:301)             contig,
    [302](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:302)             starts,
   (...)
    [307](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:307)             n_variants,
    [308](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/__init__.py:308)         )

File ~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_genotypes.py:383, in VCFGenos.read(self, contig, starts, ends, sample_idx, haplotype_idx, n_variants)
    [380](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_genotypes.py:380)     for i, v in enumerate(self.handles[contig](f"{contig}:{s}-{e}"), start=o):
    [381](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_genotypes.py:381)         if v.is_snp or v.is_indel:
    [382](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_genotypes.py:382)             # (s p)
--> [383](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_genotypes.py:383)             genos[..., i] = v.genotype.array()[:, : self.ploidy]
    [385](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_genotypes.py:385) genos = genos[_sample_idx, _haplotype_idx]
    [386](https://vscode-remote+ssh-002dremote-002bbamdev2-002ecshl-002eedu.vscode-resource.vscode-cdn.net/grid/koo/home/schilder/projects/VEP_protein/notebooks/~/.conda/envs/genome-loader/lib/python3.12/site-packages/genvarloader/_variants/_genotypes.py:386) # cyvcf2 encoding: 0, 1, -1 => gvl/pgen encoding: 0, 1, -9

IndexError: index 130628 is out of bounds for axis 2 with size 130628

Maybe bc the VCFs aren't normalised? (total guess)

[d-laub]

The categorical strand column issue should be a quick fix, it should use replace_strict and map "." to 1.

The bcftools requirement is a loose end I've missed. The error shows that GVL tries to access num_records but the VCF needs to be indexed for that property to exist, so it catches the error and tries to index it. This could be documented better. Adding bcftools as a dependency is a pain since it's a CLI tool, so it's not on PyPI and I don't know how to include it as a package dependency (sans making it a conda only package which I probably don't want to). Do you get an error if you try indexing the VCF from the CLI?

I think the VCF error you're hitting now is because there is a bug in GVL with VCFs containing indels. I didn't realize that cyvcf2 will skip variants in cases where a deletion spans the query region but downstream sites do not (fix almost done, see genoray). It should work with PGEN files right now (plink2 --make-pgen --vcf <vcf> --out <out_prefix>), or you can filter the VCF for SNPs with bcftools view -i 'TYPE="snp"'. In either case, you will need to make sure the VCF does not have any multi-allelic sites and you'll need to norm the variants since GVL does not support those. Normalization: bcftools norm -f <fasta> -a --atom-overlaps . -m - <vcf>. If you're going to rewrite the VCF anyway, I'd also recommend converting to BCF -- smaller and faster access.

[bschilder]

The categorical strand column issue should be a quick fix, it should use replace_strict and map "." to 1

Makes sense! 👍

The bcftools requirement is a loose end I've missed. The error shows that GVL tries to access num_records but the VCF needs to be indexed for that property to exist, so it catches the error and tries to index it. This could be documented better. Adding bcftools as a dependency is a pain since it's a CLI tool, so it's not on PyPI and I don't know how to include it as a package dependency (sans making it a conda only package which I probably don't want to). Do you get an error if you try indexing the VCF from the CLI?

Yeah, haven't found a great way to distribute bcftools. Looking into whether this might be a decent option:
https://github.com/DataBiosphere/xsamtools

I think the VCF error you're hitting now is because there is a bug in GVL with VCFs containing indels. I didn't realize that cyvcf2 will skip variants in cases where a deletion spans the query region (fix pending, see genoray). It should work with PGEN files right now (plink2 --make-pgen --vcf <vcf> --out <out_prefix>), or you can filter the VCF for SNPs with bcftools view -i 'TYPE="snp"'.

Ok I see, will definitely want to keep the indels in there, but can see about converting to PGEN. I can raise this as a separate issue (if it isn't already).

Though perhaps this also has something to do with the fact that these VCFs also contain structural variants (though I'm not sure how they encode this).

In either case, you will need to make sure the VCF does not have any multi-allelic sites and you'll need to norm the variants since GVL does not support those. Normalization: bcftools norm -f <fasta> -a --atom-overlaps . -m - <vcf>. If you're going to rewrite the VCF anyway, I'd also recommend converting to BCF -- smaller and faster access.

Got it, will run the norm first.

And thanks for the tip about BCF!

[d-laub]

Gotcha, my recommendation then is to run:

bcftools view -i 'TYPE="snp,indel"' -O u <file.vcf.gz> \
| bcftools norm -f <fasta> -a --atom-overlaps . -m - -O b -o <output.bcf>

plink2 --make-pgen --bcf <output.bcf> --out <prefix>

[bschilder]

bcftools view -i 'TYPE="snp,indel"' -O u <file.vcf.gz>
| bcftools norm -f -a --atom-overlaps . -m - -O b -o <output.bcf>

plink2 --make-pgen --bcf <output.bcf> --out

Running into some issues with the

module load EBModules BCFtools && \
bcftools view -i TYPE="snp,indel" -O u 1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.vcf.gz && \ bcftools norm -f edfb24b9fee5f1060c26e092a696e447-Homo_sapiens.GRCh38.dna.chromosome.22.fa.gz -a --atom-overlaps . -m - -O b -o 1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.bcf

[filter.c:3224 filters_init1] Error: the tag "snp,indel" is not defined in the VCF header

Even changing to just TYPE="snp" gives:

[filter.c:3224 filters_init1] Error: the tag "snp" is not defined in the VCF header

[d-laub]

You might need to use single quotes around the -i expression? I've never encountered this error myself unfortunately. In case it's any use, here's the docs on bcftools filter expressions. What does bcftools --version output? Maybe your path is somehow mangled and you're not using the 1.21 version that conda installed?