Tensorflow setup on RStudio/ R | CentOS

2024/11/13 14:10:03

For the last 5 days, I am trying to make Keras/Tensorflow packages work in R. I am using RStudio for installation and have used conda, miniconda, virtualenv but it crashes each time in the end. Installing a library should not be a nightmare especially when we are talking about R (one of the best statistical languages) and TensorFlow (one of the best deep learning libraries). Can someone share a reliable way to install Keras/Tensorflow on CentOS 7?

Following are the steps I am using to install tensorflow in RStudio.

Since RStudio simply crashes each time I run tensorflow::tf_config() I have no way to check what is going wrong.

enter image description here

devtools::install_github("rstudio/reticulate")
devtools::install_github("rstudio/keras") # This package also installs tensorflow
library(reticulate)
reticulate::install_miniconda()
reticulate::use_miniconda("r-reticulate")
library(tensorflow)
tensorflow::tf_config() **# Crashes at this point**sessionInfo()R version 3.6.0 (2019-04-26)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.solocale:[1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     other attached packages:
[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 loaded via a namespace (and not attached):[1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
[17] base64enc_0.1-3

Update 1 The only way RStudio does not crash while installing tensorflow is by executing following steps -

First, I created a new virtual environment using conda

conda create --name py38 python=3.8.0
conda activate py38
conda install tensorflow=2.4

Then from within RStudio, I installed reticulate and activated the virtual environment which I earlier created using conda

devtools::install_github("rstudio/reticulate")
library(reticulate)
reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
reticulate::py_available(initialize = TRUE)
ts <- reticulate::import("tensorflow")

As soon as I try to import tensorflow in RStudio, it loads the library /lib64/libstdc++.so.6 instead of /root/.conda/envs/py38/lib/libstdc++.so.6 and I get the following error -

Error in py_module_import(module, convert = convert) : ImportError: Traceback (most recent call last):File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>from tensorflow.python._pywrap_tensorflow_internal import *File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hookmodule = _import(
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)Failed to load the native TensorFlow runtime.See https://www.tensorflow.org/install/errorsfor some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

Here is what inside /lib64/libstdc++.so.6

> strings /lib64/libstdc++.so.6 | grep GLIBCGLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBC_2.3
GLIBC_2.2.5
GLIBC_2.14
GLIBC_2.4
GLIBC_2.3.2
GLIBCXX_DEBUG_MESSAGE_LENGTH

To resolve the library issue, I added the path of the correct libstdc++.so.6 library having GLIBCXX_3.4.20 in RStudio.

system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')

and, also

Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")

But still I get the same error ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20'. Somehow RStudio still loads /lib64/libstdc++.so.6 first instead of /root/.conda/envs/py38/lib/libstdc++.so.6

Instead of RStudio, if I execute the above steps in the R console, then also I get the exact same error.

Update 2: A solution is posted here

Answer

Perhaps my failed attempts will help someone else solve this problem; my approach:

  • boot up a clean CentOS 7 vm
  • install R and some dependencies
sudo yum install epel-release
sudo yum install R
sudo yum install libxml2-devel
sudo yum install openssl-devel
sudo yum install libcurl-devel
sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
  • Download and install Anaconda via linux installer script
  • Create a new conda env
conda init
conda create --name tf
conda activate tf
conda install -c conda-forge tensorflow

**From within this conda env you can import tensorflow in python without error; now to access tf via R

  • install an updated gcc via devtoolset
sudo yum install centos-release-scl
sudo yum install devtoolset-7-gcc*
  • attempt to use tensorflow in R via the reticulate package
scl enable devtoolset-7 R
install.packages("remotes")
remotes::install_github('rstudio/reticulate')
reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
reticulate::repl_python()
# This works as expected but the command "import tensorflow" crashes R
# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'# Also tried:
install.packages("devtools")
devtools::install_github('rstudio/tensorflow')
devtools::install_github('rstudio/keras')
library(tensorflow)
install_tensorflow() # "successful"
tensorflow::tf_config()
# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
  • try older versions of tensorflow/keras
devtools::install_github('rstudio/[email protected]')
devtools::install_github('rstudio/[email protected]')
library(tensorflow)
tf_config()
# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
  • Try an updated version of R (v4.0)
# deactivate conda
sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm 
export R_VERSION=4.0.0
curl -O https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm
sudo yum install R-${R_VERSION}-1-1.x86_64.rpmscl enable devtoolset-7 /opt/R/4.0.0/bin/R
install.packages("devtools")
devtools::install_github('rstudio/reticulate')
reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
reticulate::repl_python()
# 'import tensorflow' resulted in "core dumped"

I guess the issue is with R/CentOS, as you can import and use tensorflow via python normally, but I'm not sure what else to try.

I would also like to say that I had no issues with Ubuntu (which is specifically supported by tensorflow, along with macOS and Windows), and I came across these docs that might be some help: https://wiki.hpcc.msu.edu/display/ITH/Installing+TensorFlow+using+anaconda / https://wiki.hpcc.msu.edu/pages/viewpage.action?pageId=22709999

https://en.xdnf.cn/q/72083.html

Related Q&A

Cant import soundfile

Im using Anaconda and Im trying to import soundfile/pysoundfile. I installed the package by running conda install -c conda-forge pysoundfile and I think it succeeded because when I run conda list it sh…

Most efficient way to multiply a small matrix with a scalar in numpy

I have a program whose main performance bottleneck involves multiplying matrices which have one dimension of size 1 and another large dimension, e.g. 1000: large_dimension = 1000a = np.random.random((1…

MultiValueDictKeyError / request.POST

I think I hav a problem at request.POST[title]MultiValueDictKeyError at /blog/add/post/"title"Request Method: GETRequest URL: http://119.81.247.69:8000/blog/add/post/Django Version: 1.8.…

How can I auto run py.test once a relative command has been change?

Via autonose or nosy, it will automatically run the nosetests once the some tests file or the relative files have been changes. I would like to ask that whether py.test provides the similar function fo…

Publish a post using XML-RPC WordPress API and Python with category

Im doing a migration from a website to another one which use Wordpress. I created new custom types for my needs (with the plugin Custom Post Types), and I created categories for each custom type.I then…

Django registration email not sending

Ive been trying to get the django-registration-redux account activation email to send to newly registered users.Ive gotten all non-email related parts to work, such as loggin in/out and actually regist…

NumPy data type comparison

I was playing with comparing data types of two different arrays to pick one that is suitable for combining the two. I was happy to discover that I could perform comparison operations, but in the proces…

A simple method for rotating images in reportlab

How can we easily rotate an image using reportlab? I have not found an easy method. The only way found comes from http://dods.ipsl.jussieu.fr/orchidee/SANORCHIDEE/TEMP/TEMP_LOCAL/cdat_portable/lib_new…

XML header getting removed after processing with elementtree

i have an xml file and i used Elementtree to add a new tag to the xml file.My xml file before processing is as follows <?xml version="1.0" encoding="utf-8"?><PackageInfo …

How to ntp server time down to millisecond precision using Python ntplib?

I am creating a python module that will output the time from a selection of NTP Pool servers to millisecond precision as an exercise in showing how server timestamps vary. Thus far I have been able to …