The data w/ id 41949 and its checksum don't match.

[yoichii]

Description

We get a checksum error every time we try to download the dataset w/ id 41949.

Steps/Code to Reproduce

import openml
openml.datasets.get_dataset(41949)

Expected Results

Successfully get the dataset.

Actual Results

The data and its checksum don't match.

OpenMLHashException: Checksum of downloaded file is unequal to the expected checksum b1a29cd9bed190cd066f750fb8247528 when downloading https://api.openml.org/data/v1/download/21378932/Elegibilidade.arff. Raised when downloading dataset 41949.

Versions

Linux-4.15.0-192-generic-x86_64-with-glibc2.27
Python 3.9.12 (main, Apr 5 2022, 06:56:58)
[GCC 7.5.0]
NumPy 1.23.4
SciPy 1.9.3
Scikit-Learn 1.1.3
OpenML 0.12.2

[PGijsbers]

Thanks for reporting the issue. It seems to be some issue with the file encoding (openml-python assumes it is utf-8, but it is not. The file reports it is ISO-8859 but there seem to be characters that are non-compliant, so it might be some variant.), we are looking into this. In the meantime, it is still possible to download the data from openml.org/d/41949, but I don't have a clean workaround for loading it with openml-python. Stay tuned :)

[LennartPurucker]

Heyho,

so I looked into this, and so far, these are my finding:

The checksum of the server's text response is not equal to the server's checksum.
However, the checksum of the sever's (binary) content response is equal to the stored checksum.

response = session.get(url, params=data)  # Get the response from the server for this dataset (called by get_dataset)

__is_checksum_equal(response.text, md5_checksum)  # False 

md5 = hashlib.md5()
md5.update(response.content)
md5_checksum_content = md5.hexdigest()
checksum_equal_to_content = md5_checksum_content == md5_checksum   # True 

How should we go forward with this @PGijsbers?
We could handle this edge case for this specific encoding ("windows-1252") by updating the response.text parameter.

But this would then need to support any encoding and this would not be UTF-8 anymore.
Otherwise, we could still return the dataset as UTF-8 (i.e., the wrong encoding) and assume that the user of the dataset needs to handle the encoding during preprocessing.

[mfeurer]

I think at some point we agreed that all data from OpenML is UTF-8. Therefore, this would be a server issue (see also openml/OpenML#201 and openml/OpenML#177) and I believe the server needs to be updated to reject non-UTF8 uploads.

As a fix for OpenML-Python, we could check the file encoding and fail it is not UTF-8?

[LennartPurucker]

The header of the server's response also tells us that the encoding is supposed to be UTF-8: 'text/xml; charset=utf-8

Verifying the checksum using the response.text implicitly already checks the file encoding, but we won't be able to tell that the wrong encoding is the reason for a different checksum easily, I think.

Maybe we could check the checksum of response.content after response.text's checksum fails, and if it is correct, we at least know the encoding is not UTF-8 - this would give a better error message IMO. Yet, it would still be a server problem in the end.

[PGijsbers]

This issue is specifically introduced by user uploaded content, which (mostly?) should move over to Parquet anyway. I'd be fine with the two-step process that Lennart describes to at least provide a better error message for those edge cases.

[mfeurer]

Yes, @LennartPurucker, that would be good to have. I think we will have arff for the next half year or so around, so having this as a fallback would be good to have.

[LennartPurucker]

I will add a PR for this today.

[LennartPurucker]

Closing this issue at the PR should have resolved this issue from our side, and it now points to server-related problems with the error message.