How to log into Google Cloud Storage from a python function?

2024/9/16 23:09:07

I am new to google cloud storage and I try to set up a function that downloads a blob once a day. At the moment I am working in my Jupyter Notebook but finally, the code will run in an Azure Function. I am struggling with setting up the client that connects me to the bucket. I have a service account credential JSON which enables me to connect to google.

Locally I have found a solution:

from google.cloud import storageclient = storage.Client.from_service_account_json('<PATH_TO_SERVICE_ACCOUNT_JSON>')

The problem is that I do not have a path where I store my JSON in the cloud but I store it in the key vault. I came up with the following solution:

from google.cloud import storage
import json
from google.oauth2 import service_accountstring_key = get_key_from_key_vault()
service_account_info = json.loads(string_key)
google_credentials = service_account.Credentials.from_service_account_info(service_account_info
)
scoped_credentials = google_credentials.with_scopes(['https://www.googleapis.com/auth/cloud-platform.read-only'])
print(type(scoped_credentials))
client = storage.Client(credentials = scoped_credentials)

I am not totally sure if I need the scoped_credentials = ...part but I only have read permissions on the bucket. (if I skip the part the error stays the same)

When I go for this solution I get the following error:

DefaultCredentialsError: Could not automatically determine credentials. Please set 
GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. Formore information, please see https://cloud.google.com/docs/authentication/getting-started

I do not have a clue what I am doing wrong because I think that I already set the credentials explicitly.

Best P

Answer

you can set the environment variable GOOGLE_APPLICATION_CREDENTIALS with the path of the json file and authenticate your function by starting the storage client without parameters.

client = storage.Client()

*by default the storage client uses the file path on the environment variable GOOGLE_APPLICATION_CREDENTIALS

It is the easiest way to use JSON credentials and it is compatible with most of Google Cloud python libraries.

https://en.xdnf.cn/q/72588.html

Related Q&A

OpenCV Python, reading video from named pipe

I am trying to achieve results as shown on the video (Method 3 using netcat)https://www.youtube.com/watch?v=sYGdge3T30oThe point is to stream video from raspberry pi to ubuntu PC and process it using …

Load model with ML.NET saved with keras

I have a Neural Network implemented in Python with Keras. Once I have trained it I have exported the model and I have got two files: model.js and model.h5. Now I want to classify in real time inside a …

unable to add spark to PYTHONPATH

I am struggling to add spark to my python path:(myenv)me@me /home/me$ set SPARK_HOME="/home/me/spark-1.2.1-bin-hadoop2.4" (myenv)me@me /home/me$ set PYTHONPATH=$PYTHONPATH:$SPARK_HOME:$SPARK_…

Is there a way to shallow copy an existing file-object?

The use case for this would be creating multiple generators based on some file-object without any of them trampling each others read state. Originally I (thought I) had a working implementation using s…

Python - Import package modules before as well as after setup.py install

Assume a Python package (e.g., MyPackage) that consists of several modules (e.g., MyModule1.py and MyModule2.py) and a set of unittests (e.g., in MyPackage_test.py).. ├── MyPackage │ ├── __ini…

Where can I find the _sre.py python built-in module? [closed]

Closed. This question is seeking recommendations for books, tools, software libraries, and more. It does not meet Stack Overflow guidelines. It is not currently accepting answers.We don’t allow questi…

When numba is effective?

I know numba creates some overheads and in some situations (non-intensive computation) it become slower that pure python. But what I dont know is where to draw the line. Is it possible to use order of …

How to launch a Windows shortcut using Python

I want to launch a shortcut named blender.ink located at "D://games//blender.ink". I have tryed using:-os.startfile ("D://games//blender.ink")But it failed, it only launches exe fil…

Providing context in TriggerDagRunOperator

I have a dag that has been triggered by another dag. I have passed through to this dag some configuration variables via the DagRunOrder().payload dictionary in the same way the official example has don…

Install gstreamer support for opencv python package

I have built my own opencv python package from source. import cv2 print(cv2.__version__)prints: 3.4.5Now the issue I am facing is regarding the use of gstreamer from the VideoCapture class of opencv. I…