Epochly Optimization Guide: Activation Thresholds

Epochly uses empirically-validated thresholds to determine when to activate each optimization level. This document explains these thresholds and how to tune them.

Overview

Activation thresholds control when Epochly enables different optimization strategies. These thresholds are based on benchmarks across thousands of workloads to ensure optimizations only apply when they provide real benefit.

Why Thresholds Matter

Optimization has overhead:

• Thread pool creation takes ~10ms
• JIT compilation takes ~100ms for first compile
• Worker spawning takes ~500ms
• GPU initialization takes ~200ms

If the workload doesn't exceed threshold minimums, the overhead exceeds the benefit.

Default Thresholds

Level 1: Threading Thresholds

Level 2: JIT Thresholds

Level 3: Multicore Thresholds

Level 4: GPU Thresholds

Configuring Thresholds

Via Environment Variables

# Set enhancement level
export EPOCHLY_LEVEL=2
# Set operating mode
export EPOCHLY_MODE=balanced

Via Runtime API

import epochly
# Set enhancement level
epochly.configure(enhancement_level=2)
# Enable/disable JIT
epochly.configure(jit_enabled=True)
# Set GPU memory limit
epochly.configure(memory_limit=4096)

Threshold Tuning Guide

When to Lower Thresholds

Lower thresholds when:

• Your workload is known to be parallelizable
• You have dedicated compute resources
• Latency is less important than throughput
• You're willing to accept higher startup overhead

When to Raise Thresholds

Raise thresholds when:

• Running many small operations
• Low latency is critical
• Memory is constrained
• Running on shared infrastructure

Empirical Threshold Data

Level 1 Threading Benchmarks

Threading becomes beneficial when I/O wait time exceeds threading overhead:

Level 2 JIT Benchmarks

JIT compilation provides benefit after sufficient calls to amortize compile time:

Level 3 Multicore Benchmarks

Parallelization overhead depends on data size and transfer costs:

Level 4 GPU Benchmarks

GPU benefits depend heavily on data transfer costs:

Monitoring Threshold Effectiveness

Check Activation Statistics

import epochly
# Get current metrics
metrics = epochly.get_metrics()
print(f"Optimized calls: {metrics.get('optimized_calls', 0)}")
print(f"Total calls: {metrics.get('total_calls', 0)}")
print(f"Current level: {epochly.get_level()}")

Analyze Performance

import epochly
# Use monitoring context for detailed tracking
with epochly.monitoring_context():
    result = my_function(data)
# Check status after run
status = epochly.get_status()
print(f"Enhancement level: {status.get('enhancement_level')}")
print(f"Mode: {status.get('mode')}")

Best Practices

1. Start with Defaults

Epochly's defaults are based on extensive benchmarking. Only change them if you have measured evidence that different values work better for your workload.

2. Profile Before Tuning

import epochly
# Use monitoring context for profiling
with epochly.monitoring_context():
    result = my_function(data)
# Check metrics
metrics = epochly.get_metrics()
print(f"Total time: {metrics.get('total_time_ms', 0):.2f}ms")

3. Test Both Directions

When tuning, test both higher and lower enhancement levels:

import time
import epochly
for level in [0, 1, 2, 3, 4]:
    epochly.configure(enhancement_level=level)
    start = time.perf_counter()
    for _ in range(1000):
        my_function(data)
    elapsed = time.perf_counter() - start
    print(f"Level {level}: {elapsed:.3f}s")

4. Consider Workload Variability

If your workload varies significantly, use more conservative (higher) thresholds to avoid overhead on small operations.

5. Monitor in Production

Enable threshold telemetry to understand real-world activation patterns:

export EPOCHLY_TELEMETRY=true

Troubleshooting

Optimization Not Activating

1. Check current thresholds: epochly.get_config()
2. Check workload size matches threshold requirements
3. Enable debug logging: export EPOCHLY_DEBUG=true

Optimization Causing Slowdown

1. Workload may be below threshold - raise thresholds
2. Check for excessive overhead with epochly.monitoring_context()
3. Consider disabling specific levels: epochly.set_level(2)

Inconsistent Performance

1. Workload may be near threshold boundary
2. Consider raising threshold to avoid activation jitter
3. Use manual level control for critical paths