0
$\begingroup$

I have been facing difficulties while loading specific lines from a text file. The lines contain characters such as

ٹام بیمار ÛÛ’Û” ٹام بیمار ÛÛ’.

I have tried using different encodings, but I haven't been successful in retrieving the original form of the text.

The lines are still not decoded correctly, and the output contains the same encoded characters.

I suspect that the lines may be encoded in a different format or have a specific encoding requirement. Is there a way to properly load these lines from the text file and retrieve their original form?

Any suggestions or insights would be greatly appreciated. Thank you!

$\endgroup$

1 Answer 1

0
$\begingroup$

On Debian/Ubuntu or MacOs you can use uchardet to try to guess the encoding. Install it with brew install uchardet on MacOS and with sudo apt install uchardet on Debian/Ubuntu.

If uchardet is not available for your OS, you can use chardet, a python package with an accompanying executable. To use it, just install it globally with pip install chardet and then run it with chardetect <filename>.

Once you know the encoding, you can use iconv to convert it to UTF-8 or the encoding of your choice.

$\endgroup$
2
  • 1
    $\begingroup$ Thank you for the answer, surprisingly when I opened the file on MAC it worked very well as compared to Windows, even I tried to load the text on jupytrer notebook as well with standard methods, will give a try to your method to see the results $\endgroup$ Jun 8, 2023 at 15:02
  • $\begingroup$ @AbdulBasitNiazi did you have the chance to try the approach I proposed? $\endgroup$
    – noe
    Jun 19, 2023 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.