When building backend systems that track complex state—such as vehicle telemetry, user sessions, or sensor data—you often need an in-memory data store. Valkey (the open-source Redis alternative) is incredibly fast, but its native data types are limited to strings, lists, sets, and basic hashes.

If you need to store nested Python objects, custom class instances, or complex dictionaries inside a Valkey Hash, Python’s built-in pickle module is the perfect bridge.

Why Use Valkey Hashes (HSET / HGETALL)?

While you could simply serialize an entire object and store it as a single flat string key (SET user:1001 <pickled_data>), using Valkey Hashes (HSET) provides more granularity. It allows you to group related fields under a single Valkey key while still keeping the values serialized.

For example, tracking a vehicle’s state:

Key: vehicle:tracker:B1234XYZ
Fields:
  - "motion_status" -> (pickled object)
  - "sensor_data"   -> (pickled object)
  - "geofence"      -> (pickled object)

Code Example: Serialization and Deserialization

Here is how you can integrate pickle with the standard redis-py client (which fully supports Valkey) to store and retrieve data.

import pickle
import redis # Works perfectly for Valkey

# 1. Initialize Valkey client
valkey_client = redis.Redis(host='localhost', port=6379, db=0)

# 2. Define some complex Python data
vehicle_sensor_data = {
    "odometer": 150230,
    "speed": 65.5,
    "temperatures": [22.1, 23.0, 21.8],
    "active_errors": set(["E_001", "W_042"]) # Sets aren't natively JSON serializable!
}

geofence_status = {
    "zone": "Jakarta_South",
    "status": "ENTER",
    "timestamp": 1715340000
}

# 3. Serialize and save to Valkey Hash (HSET)
valkey_key = "vehicle:tracker:B1234XYZ"

valkey_client.hset(valkey_key, mapping={
    "sensor_data": pickle.dumps(vehicle_sensor_data),
    "geofence": pickle.dumps(geofence_status)
})

print("Data successfully saved to Valkey!")

# 4. Retrieve and deserialize data (HGETALL)
# HGETALL returns a dictionary of byte-strings mapping fields to values
raw_data = valkey_client.hgetall(valkey_key)

# We need to unpickle the values
restored_data = {}
for field, pickled_val in raw_data.items():
    # Convert byte-key to string, unpickle the byte-value
    field_str = field.decode('utf-8')
    restored_data[field_str] = pickle.loads(pickled_val)

print("\nRestored Data:")
print("Sensor Data:", restored_data["sensor_data"])
print("Geofence:", restored_data["geofence"])

Important Security Warning ⚠️

The pickle module is not secure against erroneously or maliciously constructed data. Unpickling data can execute arbitrary Python code.

Only unpickle data that you trust. If your Valkey instance is exposed or if you are accepting payloads from a public API, consider using json or msgpack instead. However, for internal microservices processing trusted platform data (like an internal scheduler or cron job), pickle provides an incredibly convenient way to maintain exact Python types (like datetime, set, and custom objects) across your infrastructure.