Utilities API Reference

Minimal, auto-generated docs for utility modules.

Serialization

Serialization utilities for converting data to JSON-serializable format.

scrapy_item_ingest.utils.serialization.serialize_item_data(item_dict)[source]

Serialize item data to JSON string

Fingerprint

Request fingerprint utilities for generating unique request identifiers.

scrapy_item_ingest.utils.fingerprint.get_request_fingerprint(request)[source]

Generate a fingerprint for the request

Time helpers

scrapy_item_ingest.utils.time.get_current_datetime(settings)[source]

Returns the current datetime localized to the timezone defined by settings.get_tz(). Raises a TypeError if settings is None or invalid.

Notes

  • These helpers are used by pipelines/extensions; they are safe to import in user code.

  • See quickstart and examples for practical usage.

    ‘title’: 200, ‘description’: 1000

    }

    }

    validation_result = validate_item_structure(item, schema)

Custom Serialization Classes

Request Fingerprinting

Request fingerprint utilities for generating unique request identifiers.

scrapy_item_ingest.utils.fingerprint.get_request_fingerprint(request)[source]

Generate a fingerprint for the request

Fingerprint Functions

Database Utilities

Connection Utilities

Query Builders

Data Processing

Data Cleaning

Data Validation

Caching Utilities

Redis Cache

Memory Cache

Monitoring Utilities

Performance Monitoring

Health Checks

Configuration Helpers

Environment Utilities

Testing Utilities

Test Data Generation

Database Testing

See Also