ImportError when from transformers import BertTokenizer

2024/10/18 15:00:22

My code is:

import torch
from transformers import BertTokenizer
from IPython.display import clear_output

I got error in line from transformers import BertTokenizer:

ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /mnt/home/wbj/anaconda3/envs/pytorch/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-x86_64-linux-gnu.so)

and I found an answer that the problem is due to the file /lib/x86_64-linux-gnu/libm.so.6, when I use code strings /lib/x86_64-linux-gnu/libm.so.6 | grep GLIBC_ I get the output

GLIBC_2.2.5
GLIBC_2.4
GLIBC_2.15
GLIBC_2.18
GLIBC_2.23
GLIBC_PRIVATE

The file doesn't support GLIBC_2.29. How can I fix the problem?

Answer

I had the same issue. This is due to the version of GLIBC that the tokenizers package requires

You have two options

  1. You can upgrade the glibc libraries if you have the rights

  2. The option that I have used. Installing an older version of tokenizers, for example with anaconda

In this second case, you can just run this command:

conda install -c huggingface tokenizers=0.10.1 transformers=4.6.1

Note: You can choose other versions for transformers, in this case the errors just come when you select newer versions of tokenizers

https://en.xdnf.cn/q/72729.html

Related Q&A

How to get feature names of shap_values from TreeExplainer?

I am doing a shap tutorial, and attempting to get the shap values for each person in a dataset from sklearn.model_selection import train_test_split import xgboost import shap import numpy as np import …

How can I clear a line in console after using \r and printing some text?

For my current project, there are some pieces of code that are slow and which I cant make faster. To get some feedback how much was done / has to be done, Ive created a progress snippet which you can s…

installing pyaudio to docker container

I am trying to install pyaudio to my docker container and I was wondering if anyone had any solution for Windows. I have tried two methods: Method 1: Using pipwin - Error Code: => [3/7] RUN pip inst…

Escaping special characters in elasticsearch

I am using the elasticsearch python client to make some queries to the elasticsearch instance that we are hosting.I noticed that some characters need to be escaped. Specifically, these...+ - &&…

Interacting with live matplotlib plot

Im trying to create a live plot which updates as more data is available.import os,sys import matplotlib.pyplot as pltimport time import randomdef live_plot():fig = plt.figure()ax = fig.add_subplot(111)…

pandas groupby: can I select an agg function by one level of a column MultiIndex?

I have a pandas DataFrame with a MultiIndex of columns:columns=pd.MultiIndex.from_tuples([(c, i) for c in [a, b] for i in range(3)]) df = pd.DataFrame(np.random.randn(4, 6),index=[0, 0, 1, 1],columns=c…

Bottle web app not serving static css files

My bottle web application is not serving my main.css file despite the fact I am using the static_file method.app.pyfrom bottle import * from xml.dom import minidom @route(/) def index():return template…

How to wrap text in OpenCV when I print it on an image and it exceeds the frame of the image?

I have a 1:1 ratio image and I want to make sure that if the text exceeds the frame of the image, it gets wrapped to the next line. How would I do it?I am thinking of doing an if-else block, where &qu…

pandas series filtering between values

If s is a pandas.Series, I know I can do this:b = s < 4or b = s > 0but I cant dob = 0 < s < 4orb = (0 < s) and (s < 4)What is the idiomatic pandas method for creating a boolean series…

python os.path.exists reports False when files is there

Hi have an application which is sometimes reporting that a file does not exist even when it does, I am using os.path.exists and the file is on a mounted network share. I am on OSX Yosemite, python 2.7.…