Bagaimana cara mengunggah file tar.bz2 ke wadah penyimpanan objek swift openstack menggunakan klien Python swift?

pemrograman


Saya menulis skrip Python yang menyertakan modul python-swiftclient untuk terhubung ke OpenStack Object Storage dan mengunggah beberapa file ke wadah setelah membaca dokumentasi python-swiftclient: https://docs.openstack.org/python-swiftclient/latest/ klien-api.html

Ini berfungsi dengan baik jika saya mengunggah file yang diakhiri dengan ekstensi .gz, namun saya mendapatkan beberapa kesalahan saat berhubungan dengan file terkompresi yang diakhiri dengan ekstensi .tar.bz2.

Saya telah menyertakan skrip Python dan kesalahan yang saya dapatkan setelah menjalankannya. Tolong tunjukkan di mana kesalahan saya, dan saya memerlukan bantuan dalam menyelesaikan masalah ini.

Apa yang saya coba:

ular piton
from keystoneauth1 import session
from keystoneauth1.identity import v3
from swiftclient.client import Connection
from swiftclient.client import ClientException
import gzip
import tarfile

# Create a password auth plugin
auth = v3.Password(auth_url='https://cloud.company.com:5000/v3/',
                   username='myaccount',
                   password='mypassword',
                   user_domain_name='Default',
                   project_name='myproject',
                   project_domain_name='Default')

# Create session
keystone_session = session.Session(auth=auth)

# Create swiftclient Connection
swift_conn = Connection(session=keystone_session)

# Create a new object with the contents of Netbox database backup
with gzip.open('/var/backup/netbox_backups/netbox_2024-03-16.psql.gz', 'rb') as file:
    swift_conn.put_object(
        container,
        'object_netbox_2024-03-16.psql.gz',
        contents=file,
        content_type='application/gzip'
    )

# Confirm the presence of the object holding the Netbox database backup
obj1 = 'object_netbox_2024-03-16.psql.gz'
container = 'netbox-backups'
try:
    resp_headers = swift_conn.head_object(container, obj1)
    print("The object " + obj1 + " was successfully created")
except ClientException as e:
    if e.http_status == '404':
        print("The object " + obj1 + " was not found!")
    else:
        print("An error occurred checking for the existence of the object " + obj1)

# Create a new object with the contents of the compressed Netbox media backup
with tarfile.open("/var/backup/netbox_backups/netbox_media_2024-03-16.tar.bz2", "r:bz2") as file_tar_bz2:
    swift_conn.put_object(
        container,
        'object_netbox_media_2024-03-16.tar.bz2',
        contents=file_tar_bz2,
        content_type='application/x-tar'
    )

# Confirm the presence of the object holding the compressed Netbox media backup
obj2 = 'object_netbox_media_2024-03-16.tar.bz2'
container = 'netbox-backups'
try:
    resp_headers = swift_conn.head_object(container, obj2)
    print("The object " + obj2 + " was successfully created")
except ClientException as e:
    if e.http_status == '404':
        print("The object " + obj2 + " was not found!")
    else:
        print("An error occurred checking for the existence of the object " + obj2)

Di bawah ini adalah kesalahan yang saya dapatkan setelah menjalankan skrip

ular piton
(venv) [root@scs-sandbox01 backups]# python netbox_backups_transfer.py 

The object object_netbox_2024-03-16.psql.gz was successfully created
Traceback (most recent call last):
  File "/opt/scripts/netbox_backups_transfer.py", line 52, in <module>
    swift_conn.put_object(
  File "/opt/netbox/venv/lib64/python3.9/site-packages/swiftclient/client.py", line 1963, in put_object
    return self._retry(reset_func, put_object, container, obj, contents,
  File "/opt/netbox/venv/lib64/python3.9/site-packages/swiftclient/client.py", line 1797, in _retry
    rv = func(self.url, self.token, *args,
  File "/opt/netbox/venv/lib64/python3.9/site-packages/swiftclient/client.py", line 1421, in put_object
    conn.request('PUT', path, contents, headers)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/swiftclient/client.py", line 416, in request
    self.resp = self._request(method, url, headers=headers, data=data,
  File "/opt/netbox/venv/lib64/python3.9/site-packages/swiftclient/client.py", line 400, in _request
    return self.request_session.request(*arg, **kwarg)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/opt/netbox/venv/lib64/python3.9/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/opt/netbox/venv/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
  File "/opt/netbox/venv/lib64/python3.9/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/opt/netbox/venv/lib64/python3.9/site-packages/urllib3/connection.py", line 399, in request
    for chunk in chunks:
  File "/opt/netbox/venv/lib64/python3.9/site-packages/swiftclient/utils.py", line 443, in iter_wrapper
    if len(chunk) == 0:
TypeError: object of type 'TarInfo' has no len()

Solusi 1

Masalah Anda terletak pada pesan –

Mengutip:

TypeError: objek atau tipe ‘TarInfo’ tidak memiliki len()

Fungsi ‘iter_wrapper’ Anda di modul ‘swiftclient.utils’ mencoba membaca panjang sebagian data, tetapi malah menemukan objek ‘TarInfo’, yang tidak memiliki panjang yang ditentukan atau ditetapkan saat Anda meneruskan objek ‘tarfile.TarFile’ langsung ke ‘swift_conn.put_object’ sebagai parameter konten. Ini tidak akan berfungsi karena metode ‘put_object’ Anda mengharapkan objek seperti file dengan metode ‘read()’.

Anda perlu membaca konten file tar.bz2 dan kemudian meneruskannya sebagai objek mirip file ke ‘put_object’ – Metode baca File Python ().[^]

ular piton
with tarfile.open("/var/backup/netbox_backups/netbox_media_2024-03-16.tar.bz2", "r:bz2") as file_tar_bz2:
#Read the contents...
    file_contents = file_tar_bz2.read()

#Create a file-like object from the contents...
my_file_like_object = io.BytesIO(file_contents)

#Upload the returned contents to Swift...
swift_conn.put_object(
    container,
    'object_netbox_media_2024-03-16.tar.bz2',
    contents=my_file_like_object,
    content_type='application/x-tar'
)

コメント

タイトルとURLをコピーしました