New doc supporting CORS, gRCP, and general client integration
This commit is contained in:
@@ -0,0 +1,344 @@
|
||||
# Notifier Service Client Integration Prompt
|
||||
|
||||
**Use this prompt with Claude or another AI assistant when building a client application that integrates with the Notifier service.**
|
||||
|
||||
---
|
||||
|
||||
## Prompt for AI Assistant
|
||||
|
||||
I need to build a client application that integrates with a notification service using gRPC. Please help me implement a production-ready client that follows best practices for performance, security, and reliability.
|
||||
|
||||
### Service Details
|
||||
|
||||
**Protocol:** gRPC (HTTP/2)
|
||||
**Service Address:** `notifier-grpc:50051` (Kubernetes internal DNS)
|
||||
**Authentication:** API Key via gRPC metadata
|
||||
**API Key Format:** `nk_` prefix followed by 64 hex characters (e.g., `nk_abc123...`)
|
||||
|
||||
### Authentication Requirements
|
||||
|
||||
1. **API Key Delivery:**
|
||||
- Send API key in gRPC metadata with EVERY request
|
||||
- Two accepted header formats:
|
||||
- `authorization: Bearer nk_your_api_key_here` (preferred)
|
||||
- `x-api-key: nk_your_api_key_here`
|
||||
|
||||
2. **Per-Request Authentication:**
|
||||
- The service validates the API key on every RPC call
|
||||
- Authentication is NOT done at connection time
|
||||
- This means you can (and should) use long-lived connections
|
||||
|
||||
3. **Rate Limiting:**
|
||||
- Rate limits are enforced per API key
|
||||
- Default: 100 requests per minute (configurable per key)
|
||||
- Rate limit exceeded returns: `codes.ResourceExhausted`
|
||||
- Implement exponential backoff retry for rate limit errors
|
||||
|
||||
4. **Credential Management:**
|
||||
- Load API key from environment variable: `NOTIFIER_API_KEY`
|
||||
- NEVER hardcode API keys in source code
|
||||
- NEVER log the full API key (mask it: `nk_abc...xyz`)
|
||||
- Store in Kubernetes Secret if deploying to k8s
|
||||
|
||||
### Connection Management Requirements
|
||||
|
||||
**CRITICAL:** Use long-lived gRPC connections for optimal performance.
|
||||
|
||||
1. **Connection Lifecycle:**
|
||||
- Create ONE connection at application startup
|
||||
- Reuse the connection for ALL requests
|
||||
- Close the connection only when application exits
|
||||
- Do NOT create a new connection for each request
|
||||
|
||||
2. **Keepalive Configuration:**
|
||||
- Configure client keepalive to prevent connection timeouts
|
||||
- Recommended settings:
|
||||
- Time: 10 seconds (ping every 10s of inactivity)
|
||||
- Timeout: 3 seconds (wait for ping response)
|
||||
- PermitWithoutStream: true (allow pings when idle)
|
||||
|
||||
3. **Why This Matters:**
|
||||
- Connection reuse: 10x faster (1.5ms vs 15ms per request)
|
||||
- Eliminates TCP + TLS handshake overhead (10-15ms per request)
|
||||
- HTTP/2 multiplexing: handle multiple concurrent requests on one connection
|
||||
- Lower resource usage: one connection vs thousands
|
||||
|
||||
### Error Handling Requirements
|
||||
|
||||
1. **Handle These Error Codes:**
|
||||
- `codes.Unauthenticated` - Invalid or missing API key → Log error, don't retry
|
||||
- `codes.ResourceExhausted` - Rate limit exceeded → Retry with exponential backoff
|
||||
- `codes.InvalidArgument` - Bad request payload → Log error, don't retry
|
||||
- `codes.Unavailable` - Service temporarily unavailable → Retry with backoff
|
||||
- `codes.DeadlineExceeded` - Request timeout → Retry (may be transient)
|
||||
|
||||
2. **Retry Strategy:**
|
||||
- Implement exponential backoff: 100ms, 200ms, 400ms, 800ms
|
||||
- Maximum 3-5 retry attempts for retryable errors
|
||||
- Only retry: rate limits, unavailable, deadline exceeded
|
||||
- Do NOT retry: authentication errors, invalid arguments
|
||||
|
||||
3. **Context Timeouts:**
|
||||
- Set reasonable timeout for each request (5-30 seconds)
|
||||
- Use `context.WithTimeout()` for every RPC call
|
||||
- Allow timeout to be configurable
|
||||
|
||||
### Code Structure Requirements
|
||||
|
||||
1. **Client Structure:**
|
||||
```
|
||||
NotifierClient struct:
|
||||
- conn: *grpc.ClientConn (long-lived)
|
||||
- client: pb.NotifierServiceClient
|
||||
- apiKey: string (loaded from env)
|
||||
- logger: logging interface
|
||||
```
|
||||
|
||||
2. **Required Methods:**
|
||||
- `NewNotifierClient(address, apiKey string) (*NotifierClient, error)` - Initialize
|
||||
- `SendNotification(ctx, request) (response, error)` - Send single notification
|
||||
- `SendBatchNotifications(ctx, requests) (responses, error)` - Send batch
|
||||
- `HealthCheck(ctx) error` - Verify connection health
|
||||
- `Close() error` - Clean up connection
|
||||
|
||||
3. **Initialization Pattern:**
|
||||
- Create client as singleton at application startup
|
||||
- Use `sync.Once` to ensure only one instance
|
||||
- Defer `Close()` in main() to ensure cleanup
|
||||
|
||||
### Observability Requirements
|
||||
|
||||
1. **Logging:**
|
||||
- Log connection establishment (info level)
|
||||
- Log authentication failures (warn level)
|
||||
- Log rate limit exceeded (warn level)
|
||||
- Log successful sends (debug level)
|
||||
- NEVER log the full API key (mask it)
|
||||
|
||||
2. **Metrics (if using Prometheus):**
|
||||
- Counter: `notifier_requests_total` (labels: status, method)
|
||||
- Counter: `notifier_requests_failed_total` (labels: error_code)
|
||||
- Histogram: `notifier_request_duration_seconds`
|
||||
- Counter: `notifier_rate_limit_errors_total`
|
||||
|
||||
3. **Health Checks:**
|
||||
- Implement background health check goroutine
|
||||
- Check every 30-60 seconds
|
||||
- Call the service's HealthCheck RPC
|
||||
- Log warnings if health checks fail
|
||||
|
||||
### Security Requirements
|
||||
|
||||
1. **Credential Security:**
|
||||
- Load API key ONLY from environment variables
|
||||
- Never commit API keys to version control
|
||||
- Use `.gitignore` to exclude any files with credentials
|
||||
- Implement a `String()` method that masks the API key for logging
|
||||
|
||||
2. **TLS Configuration:**
|
||||
- For local development: `grpc.WithInsecure()` is acceptable
|
||||
- For production: Use TLS with proper certificate validation
|
||||
- For Kubernetes: Internal communication may use insecure (cluster network is trusted)
|
||||
|
||||
3. **Graceful Degradation:**
|
||||
- If notifier service is unavailable, application should continue
|
||||
- Log notification failures but don't crash the application
|
||||
- Consider implementing a circuit breaker pattern for resilience
|
||||
|
||||
### Testing Requirements
|
||||
|
||||
1. **Unit Tests:**
|
||||
- Mock the gRPC client interface
|
||||
- Test retry logic with rate limit errors
|
||||
- Test context timeout handling
|
||||
- Test credential masking in logs
|
||||
|
||||
2. **Integration Tests:**
|
||||
- Test actual connection to notifier service (if available)
|
||||
- Test authentication with valid and invalid keys
|
||||
- Test rate limiting behavior
|
||||
|
||||
### Kubernetes Deployment Requirements (if applicable)
|
||||
|
||||
1. **Environment Variables:**
|
||||
```yaml
|
||||
env:
|
||||
- name: NOTIFIER_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: notifier-credentials
|
||||
key: api-key
|
||||
- name: NOTIFIER_ADDRESS
|
||||
value: "notifier-grpc:50051"
|
||||
```
|
||||
|
||||
2. **Secrets Configuration:**
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: notifier-credentials
|
||||
type: Opaque
|
||||
stringData:
|
||||
api-key: nk_your_actual_api_key_here
|
||||
```
|
||||
|
||||
### Available gRPC Methods
|
||||
|
||||
The NotifierService provides these RPC methods:
|
||||
|
||||
1. **SendNotification** - Send a single notification
|
||||
- Request: `SendNotificationRequest`
|
||||
- Response: `SendNotificationResponse` (contains notification_id)
|
||||
|
||||
2. **SendBatchNotifications** - Send multiple notifications at once
|
||||
- Request: `SendBatchNotificationsRequest`
|
||||
- Response: `SendBatchNotificationsResponse` (contains notification_ids)
|
||||
|
||||
3. **GetNotification** - Retrieve notification status by ID
|
||||
- Request: `GetNotificationRequest` (notification_id)
|
||||
- Response: `GetNotificationResponse`
|
||||
|
||||
4. **ListNotifications** - List recent notifications
|
||||
- Request: `ListNotificationsRequest` (pagination)
|
||||
- Response: `ListNotificationsResponse`
|
||||
|
||||
5. **CancelNotification** - Cancel a pending notification
|
||||
- Request: `CancelNotificationRequest` (notification_id)
|
||||
- Response: `CancelNotificationResponse`
|
||||
|
||||
6. **RetryNotification** - Retry a failed notification
|
||||
- Request: `RetryNotificationRequest` (notification_id)
|
||||
- Response: `RetryNotificationResponse`
|
||||
|
||||
7. **GetStats** - Get service statistics
|
||||
- Request: `GetStatsRequest`
|
||||
- Response: `GetStatsResponse`
|
||||
|
||||
8. **GetNotifiers** - List available notifier types and accounts
|
||||
- Request: `GetNotifiersRequest`
|
||||
- Response: `GetNotifiersResponse`
|
||||
|
||||
9. **HealthCheck** - Verify service health
|
||||
- Request: `HealthCheckRequest`
|
||||
- Response: `HealthCheckResponse`
|
||||
|
||||
### Notification Types Supported
|
||||
|
||||
- `NOTIFICATION_TYPE_EMAIL` - Email notifications
|
||||
- `NOTIFICATION_TYPE_SLACK` - Slack messages
|
||||
- `NOTIFICATION_TYPE_NTFY` - Ntfy.sh push notifications
|
||||
- `NOTIFICATION_TYPE_STDOUT` - Console output (development only)
|
||||
|
||||
### Request Example Structure
|
||||
|
||||
```protobuf
|
||||
message SendNotificationRequest {
|
||||
NotificationType type = 1; // Required: email, slack, ntfy, stdout
|
||||
string subject = 2; // Required: notification subject/title
|
||||
string body = 3; // Required: notification body/content
|
||||
repeated string recipients = 4; // Required: email addresses, slack channels, etc.
|
||||
Priority priority = 5; // Optional: normal, high, critical
|
||||
string account = 6; // Optional: specific notifier account to use
|
||||
map<string, string> metadata = 7; // Optional: additional metadata
|
||||
}
|
||||
```
|
||||
|
||||
### Implementation Goals
|
||||
|
||||
Please implement a client that:
|
||||
|
||||
1. ✅ Uses a single long-lived gRPC connection
|
||||
2. ✅ Sends API key in metadata with every request
|
||||
3. ✅ Implements exponential backoff retry for rate limits
|
||||
4. ✅ Loads credentials from environment variables
|
||||
5. ✅ Uses context with timeout for all requests
|
||||
6. ✅ Implements keepalive to prevent connection timeouts
|
||||
7. ✅ Includes proper error handling for all error codes
|
||||
8. ✅ Logs important events without exposing credentials
|
||||
9. ✅ Provides a health check mechanism
|
||||
10. ✅ Includes graceful shutdown (connection cleanup)
|
||||
11. ✅ Is production-ready with proper error handling and logging
|
||||
12. ✅ Includes basic unit tests
|
||||
|
||||
### Example Usage Pattern
|
||||
|
||||
The client should be usable like this:
|
||||
|
||||
```go
|
||||
// Initialize once at startup
|
||||
client, err := notifier.NewClient(
|
||||
os.Getenv("NOTIFIER_ADDRESS"),
|
||||
os.Getenv("NOTIFIER_API_KEY"),
|
||||
)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
// Use throughout application lifetime
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
resp, err := client.SendNotification(ctx, ¬ifier.SendRequest{
|
||||
Type: notifier.TypeEmail,
|
||||
Subject: "Important Alert",
|
||||
Body: "This is a critical notification",
|
||||
Recipients: []string{"admin@example.com"},
|
||||
Priority: notifier.PriorityHigh,
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
log.Printf("Failed to send notification: %v", err)
|
||||
// Application continues - notification failure is not fatal
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("Notification sent successfully: %s", resp.NotificationID)
|
||||
```
|
||||
|
||||
### Additional Context
|
||||
|
||||
- This is a microservices architecture running in Kubernetes
|
||||
- The notifier service handles routing to multiple notification backends (SMTP, Slack, Ntfy)
|
||||
- The client may need to send notifications from multiple goroutines concurrently
|
||||
- High reliability is important, but notification failures should not crash the application
|
||||
- The API key has a rate limit of 100 requests per minute (may vary per key)
|
||||
|
||||
### Code Generation Note
|
||||
|
||||
If you need the protobuf definitions, they can be generated from the service's proto files located at `api/grpc/proto/notifier.proto` in the notifier service repository.
|
||||
|
||||
---
|
||||
|
||||
## Usage Instructions
|
||||
|
||||
**Copy the prompt above and provide it to your AI assistant when building the client. The prompt includes:**
|
||||
|
||||
- All authentication requirements and formats
|
||||
- Connection management best practices
|
||||
- Error handling strategies
|
||||
- Security requirements
|
||||
- Observability guidelines
|
||||
- Complete API method listing
|
||||
- Production-ready patterns
|
||||
- Kubernetes deployment configuration
|
||||
|
||||
**The AI assistant should generate:**
|
||||
- Complete, production-ready client code
|
||||
- Proper connection pooling and keepalive
|
||||
- Comprehensive error handling
|
||||
- Secure credential management
|
||||
- Health check implementation
|
||||
- Unit tests
|
||||
- Usage documentation
|
||||
|
||||
**Review the generated code for:**
|
||||
- ✅ Single long-lived connection (not creating connections per request)
|
||||
- ✅ API key sent in metadata with every RPC
|
||||
- ✅ Keepalive configuration present
|
||||
- ✅ Exponential backoff retry logic
|
||||
- ✅ No hardcoded credentials
|
||||
- ✅ Proper context timeouts
|
||||
- ✅ Error handling for all gRPC error codes
|
||||
- ✅ Graceful shutdown (Close() method)
|
||||
@@ -0,0 +1,255 @@
|
||||
# CORS Implementation - Security Enhancement
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the implementation of a secure CORS (Cross-Origin Resource Sharing) configuration system that replaces the previous wildcard (`*`) configuration with an explicit origin whitelist to prevent CSRF attacks.
|
||||
|
||||
## Implementation Summary
|
||||
|
||||
### 1. CORSConfig Structure (api/rest/router.go)
|
||||
|
||||
Created a comprehensive `CORSConfig` struct with the following fields:
|
||||
|
||||
```go
|
||||
type CORSConfig struct {
|
||||
AllowedOrigins []string // Whitelist of allowed origins
|
||||
AllowedMethods []string // Allowed HTTP methods
|
||||
AllowedHeaders []string // Allowed HTTP headers
|
||||
AllowCredentials bool // Whether to allow credentials
|
||||
MaxAge int // Cache duration for preflight responses
|
||||
}
|
||||
```
|
||||
|
||||
**Key Security Features:**
|
||||
- Wildcards (`*`) are explicitly NOT supported
|
||||
- Origins must be explicitly whitelisted
|
||||
- Credentials can only be enabled with specific origins
|
||||
|
||||
### 2. CORS Middleware Implementation
|
||||
|
||||
The `newCORSMiddleware()` function implements:
|
||||
|
||||
- **Origin Validation**: Checks incoming `Origin` header against whitelist
|
||||
- **Exact Match Required**: Only sets CORS headers if origin is in whitelist
|
||||
- **Never Uses Wildcard**: Always returns the exact origin, never `*`
|
||||
- **Preflight Handling**: Properly handles OPTIONS requests
|
||||
- **Configurable Headers**: All CORS headers are configurable
|
||||
|
||||
**Code Location:** `api/rest/router.go:98-149`
|
||||
|
||||
### 3. Configuration System
|
||||
|
||||
#### Config Structure (internal/config/config.go)
|
||||
|
||||
Added `CORSConfig` to the main application configuration:
|
||||
|
||||
```go
|
||||
type CORSConfig struct {
|
||||
AllowedOrigins []string `mapstructure:"allowed_origins"`
|
||||
AllowedMethods []string `mapstructure:"allowed_methods"`
|
||||
AllowedHeaders []string `mapstructure:"allowed_headers"`
|
||||
AllowCredentials bool `mapstructure:"allow_credentials"`
|
||||
MaxAge int `mapstructure:"max_age"`
|
||||
}
|
||||
```
|
||||
|
||||
#### Default Values (internal/config/config.go:205-210)
|
||||
|
||||
```go
|
||||
AllowedOrigins: []string{} // Empty by default
|
||||
AllowedMethods: []string{"GET", "POST", "OPTIONS", "DELETE"} // Standard REST methods
|
||||
AllowedHeaders: []string{"Content-Type", "Authorization"} // Common headers
|
||||
AllowCredentials: false // Disabled by default
|
||||
MaxAge: 3600 // 1 hour
|
||||
```
|
||||
|
||||
### 4. Configuration Validation (internal/config/config.go:263-282)
|
||||
|
||||
Implemented `validateCORS()` that enforces:
|
||||
|
||||
1. **Wildcard Rejection**: `*` is not allowed in `allowed_origins`
|
||||
2. **Origin Format Validation**: Origins must start with `http://` or `https://`
|
||||
3. **Credentials Validation**: `allow_credentials` requires at least one origin
|
||||
|
||||
**Example Error Messages:**
|
||||
- `"wildcard (*) is not allowed in CORS allowed_origins for security reasons - specify exact origins instead"`
|
||||
- `"invalid origin format: example.com - origins must start with http:// or https://"`
|
||||
- `"allow_credentials is enabled but no origins are allowed - this configuration is ineffective"`
|
||||
|
||||
### 5. Updated Server Initialization (cmd/server/main.go:288-326)
|
||||
|
||||
The `startRESTServer()` function now:
|
||||
|
||||
1. Converts config CORS to `rest.CORSConfig`
|
||||
2. Logs CORS configuration on startup
|
||||
3. Warns if no origins are configured
|
||||
4. Passes CORS config to router
|
||||
|
||||
**Example Log Output:**
|
||||
```
|
||||
CORS enabled for origins: [http://localhost:3000 http://localhost:8080]
|
||||
```
|
||||
|
||||
Or:
|
||||
```
|
||||
CORS has no allowed origins configured - all cross-origin requests will be blocked
|
||||
```
|
||||
|
||||
### 6. Configuration Example (config.yaml:112-146)
|
||||
|
||||
#### Development Configuration
|
||||
```yaml
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "http://localhost:3000" # React/Next.js dev port
|
||||
- "http://localhost:8080" # Vue/Angular dev port
|
||||
- "http://localhost:5173" # Vite dev server
|
||||
allowed_methods:
|
||||
- "GET"
|
||||
- "POST"
|
||||
- "OPTIONS"
|
||||
- "DELETE"
|
||||
allowed_headers:
|
||||
- "Content-Type"
|
||||
- "Authorization"
|
||||
allow_credentials: false
|
||||
max_age: 3600
|
||||
```
|
||||
|
||||
#### Production Configuration (Example)
|
||||
```yaml
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "https://app.example.com"
|
||||
- "https://dashboard.example.com"
|
||||
- "https://api-docs.example.com"
|
||||
allowed_methods:
|
||||
- "GET"
|
||||
- "POST"
|
||||
- "OPTIONS"
|
||||
- "DELETE"
|
||||
allowed_headers:
|
||||
- "Content-Type"
|
||||
- "Authorization"
|
||||
allow_credentials: true # Enable for auth tokens
|
||||
max_age: 3600
|
||||
```
|
||||
|
||||
## Test Coverage
|
||||
|
||||
### 1. CORS Middleware Tests (api/rest/cors_test.go)
|
||||
|
||||
**Test Cases:**
|
||||
- ✅ `TestCORSMiddleware_AllowedOrigin`: Verifies allowed origins are accepted
|
||||
- ✅ `TestCORSMiddleware_BlockedOrigin`: Verifies non-whitelisted origins are rejected
|
||||
- ✅ `TestCORSMiddleware_PreflightRequest`: Tests OPTIONS preflight handling
|
||||
- ✅ `TestCORSMiddleware_Credentials`: Verifies credential header handling
|
||||
- ✅ `TestCORSMiddleware_NoWildcard`: Ensures wildcard is never returned
|
||||
- ✅ `TestCORSMiddleware_EmptyConfig`: Tests secure default (no origins)
|
||||
- ✅ `TestCORSMiddleware_MaxAge`: Tests cache duration configuration
|
||||
- ✅ `TestDefaultCORSConfig`: Verifies default configuration values
|
||||
|
||||
**Total: 8 test functions, 21 test cases**
|
||||
|
||||
### 2. CORS Validation Tests (internal/config/cors_test.go)
|
||||
|
||||
**Test Cases:**
|
||||
- ✅ `TestValidateCORS_WildcardRejection`: Wildcard origins are rejected
|
||||
- ✅ `TestValidateCORS_InvalidOriginFormat`: Invalid origin formats are rejected
|
||||
- ✅ `TestValidateCORS_CredentialsWithoutOrigins`: Credentials require origins
|
||||
- ✅ `TestValidateCORS_ValidConfigurations`: Valid configs are accepted
|
||||
- ✅ `TestValidateCORS_MultipleOrigins`: Multiple origins with wildcard rejected
|
||||
- ✅ `TestValidateCORS_EdgeCases`: Edge cases handled correctly
|
||||
|
||||
**Total: 6 test functions, 16 test cases**
|
||||
|
||||
## Security Improvements
|
||||
|
||||
### Before
|
||||
```go
|
||||
w.Header().Set("Access-Control-Allow-Origin", "*") // ❌ CSRF Vulnerability
|
||||
```
|
||||
|
||||
### After
|
||||
```go
|
||||
// Only set CORS headers if origin is in whitelist
|
||||
if allowed {
|
||||
w.Header().Set("Access-Control-Allow-Origin", origin) // ✅ Exact origin
|
||||
}
|
||||
// Otherwise, no CORS headers are set (browser blocks the response)
|
||||
```
|
||||
|
||||
## Security Guarantees
|
||||
|
||||
1. **No Wildcard**: The system makes it impossible to configure wildcard CORS
|
||||
2. **Explicit Whitelist**: All allowed origins must be explicitly configured
|
||||
3. **Validation at Startup**: Invalid configurations are rejected before the server starts
|
||||
4. **Default Secure**: Empty origin list by default (most restrictive)
|
||||
5. **Environment-Specific**: Different configs for dev/staging/production
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### For Development
|
||||
|
||||
Update your `config.yaml` to include localhost origins:
|
||||
|
||||
```yaml
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "http://localhost:3000"
|
||||
- "http://localhost:8080"
|
||||
```
|
||||
|
||||
### For Production
|
||||
|
||||
Configure your production origins:
|
||||
|
||||
```yaml
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "https://yourdomain.com"
|
||||
- "https://app.yourdomain.com"
|
||||
allow_credentials: true
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
You can also configure CORS via environment variables:
|
||||
|
||||
```bash
|
||||
export NOTIFIER_CORS_ALLOWED_ORIGINS="https://app.example.com,https://dashboard.example.com"
|
||||
export NOTIFIER_CORS_ALLOW_CREDENTIALS=true
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
✅ **CORS whitelist fully configurable** - via config.yaml or environment variables
|
||||
✅ **Wildcard configuration is impossible** - validation rejects `*` at startup
|
||||
✅ **Security headers properly set** - only for whitelisted origins
|
||||
✅ **Environment-specific configs work** - examples provided for dev/prod
|
||||
✅ **No CSRF vulnerability** - wildcard eliminated, exact origins only
|
||||
✅ **Comprehensive tests** - 37 test cases covering all scenarios
|
||||
|
||||
## Files Modified
|
||||
|
||||
1. `api/rest/router.go` - CORS config struct and middleware
|
||||
2. `api/rest/cors_test.go` - CORS middleware tests (NEW)
|
||||
3. `internal/config/config.go` - CORS configuration structure
|
||||
4. `internal/config/cors_test.go` - CORS validation tests (NEW)
|
||||
5. `cmd/server/main.go` - Server initialization with CORS config
|
||||
6. `config.yaml` - Example CORS configuration
|
||||
7. `internal/auth/bootstrap.go` - Fixed unrelated string formatting bug
|
||||
8. `internal/auth/keystore_hybrid.go` - Fixed unrelated rate limiter signature
|
||||
|
||||
## Additional Fixes
|
||||
|
||||
While implementing CORS, also fixed pre-existing build errors:
|
||||
- String multiplication in `bootstrap.go` (changed to `strings.Repeat`)
|
||||
- Rate limiter signature mismatch in `keystore_hybrid.go`
|
||||
- Missing parameter in `service_retention_test.go`
|
||||
|
||||
## References
|
||||
|
||||
- **OWASP CORS**: https://owasp.org/www-community/attacks/csrf
|
||||
- **MDN CORS**: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
|
||||
- **RFC 6454**: The Web Origin Concept
|
||||
@@ -0,0 +1,357 @@
|
||||
# CORS Configuration for Kubernetes Deployments
|
||||
|
||||
## Important: CORS Only Affects REST API, Not gRPC
|
||||
|
||||
**Key Point:** CORS (Cross-Origin Resource Sharing) is a **browser security mechanism** that only applies to HTTP/REST APIs accessed from web browsers. It does **NOT** affect gRPC communication.
|
||||
|
||||
## Service-to-Service Communication (gRPC)
|
||||
|
||||
### ✅ No CORS Configuration Needed
|
||||
|
||||
For services running in the same Kubernetes cluster communicating via gRPC:
|
||||
|
||||
**You don't need to configure CORS at all!**
|
||||
|
||||
Here's why:
|
||||
|
||||
1. **gRPC uses HTTP/2** - CORS is not enforced
|
||||
2. **Server-to-Server** - CORS only applies to browser-based requests
|
||||
3. **No Origin header** - Backend services don't send Origin headers
|
||||
|
||||
### gRPC Client Connection Example
|
||||
|
||||
```go
|
||||
// Service A connecting to notifier service via gRPC
|
||||
conn, err := grpc.Dial("notifier-grpc.default.svc.cluster.local:50051",
|
||||
grpc.WithInsecure())
|
||||
if err != nil {
|
||||
log.Fatalf("Failed to connect: %v", err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
client := pb.NewNotifierServiceClient(conn)
|
||||
```
|
||||
|
||||
**No CORS configuration required** ✅
|
||||
|
||||
The gRPC service runs on port `50051` (as defined in your k8s/service.yaml) and is accessible at:
|
||||
- From same namespace: `notifier-grpc:50051`
|
||||
- From other namespace: `notifier-grpc.default.svc.cluster.local:50051`
|
||||
- Full DNS: `notifier-grpc.default.svc.cluster.local:50051`
|
||||
|
||||
## When You DO Need CORS Configuration
|
||||
|
||||
CORS configuration is **only** needed when:
|
||||
|
||||
1. **Web browsers** access the REST API (port 8080)
|
||||
2. The web app is served from a **different origin** than the API
|
||||
3. The request goes through the **REST API**, not gRPC
|
||||
|
||||
### Common Scenarios
|
||||
|
||||
| Scenario | Needs CORS? | Reason |
|
||||
|----------|-------------|---------|
|
||||
| Backend service → gRPC API | ❌ No | Not a browser, uses gRPC |
|
||||
| Backend service → REST API | ❌ No | Not a browser |
|
||||
| Web app → gRPC (grpc-web) | ✅ Yes | Browser-based, needs CORS |
|
||||
| Web app → REST API | ✅ Yes | Browser-based, needs CORS |
|
||||
| CLI tool → REST API | ❌ No | Not a browser |
|
||||
| Postman/curl → REST API | ❌ No | Not a browser |
|
||||
|
||||
## CORS Configuration for Web Applications
|
||||
|
||||
If you have a web frontend that calls the REST API, you need to configure CORS.
|
||||
|
||||
### Scenario 1: Frontend in Same Cluster
|
||||
|
||||
**Example:** Frontend at `https://app.example.com`, API at `https://notifier.example.com`
|
||||
|
||||
Update your ConfigMap (k8s/configmap.yaml):
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: notifier-config
|
||||
data:
|
||||
config.yaml: |
|
||||
# ... existing config ...
|
||||
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "https://app.example.com" # Your frontend domain
|
||||
allowed_methods:
|
||||
- "GET"
|
||||
- "POST"
|
||||
- "OPTIONS"
|
||||
- "DELETE"
|
||||
allowed_headers:
|
||||
- "Content-Type"
|
||||
- "Authorization"
|
||||
allow_credentials: true # If your frontend sends auth tokens
|
||||
max_age: 3600
|
||||
```
|
||||
|
||||
### Scenario 2: Multiple Frontends
|
||||
|
||||
```yaml
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "https://app.example.com" # Main app
|
||||
- "https://dashboard.example.com" # Admin dashboard
|
||||
- "https://mobile.example.com" # Mobile web app
|
||||
allowed_methods:
|
||||
- "GET"
|
||||
- "POST"
|
||||
- "OPTIONS"
|
||||
- "DELETE"
|
||||
allowed_headers:
|
||||
- "Content-Type"
|
||||
- "Authorization"
|
||||
allow_credentials: true
|
||||
max_age: 3600
|
||||
```
|
||||
|
||||
### Scenario 3: Development Environment
|
||||
|
||||
For local development (frontend at http://localhost:3000):
|
||||
|
||||
```yaml
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "http://localhost:3000" # React/Next.js dev server
|
||||
- "http://localhost:8080" # Vue/Angular dev server
|
||||
- "http://localhost:5173" # Vite dev server
|
||||
- "https://app.example.com" # Production frontend
|
||||
allowed_methods:
|
||||
- "GET"
|
||||
- "POST"
|
||||
- "OPTIONS"
|
||||
- "DELETE"
|
||||
allowed_headers:
|
||||
- "Content-Type"
|
||||
- "Authorization"
|
||||
allow_credentials: false
|
||||
max_age: 3600
|
||||
```
|
||||
|
||||
### Scenario 4: No Web Frontend (Backend Services Only)
|
||||
|
||||
If you only have backend services using gRPC:
|
||||
|
||||
```yaml
|
||||
cors:
|
||||
# Empty or minimal config - no origins needed
|
||||
allowed_origins: [] # No browser clients
|
||||
allowed_methods:
|
||||
- "GET"
|
||||
- "POST"
|
||||
allowed_headers:
|
||||
- "Content-Type"
|
||||
```
|
||||
|
||||
The REST API will still work for non-browser clients (like curl, Postman, or backend HTTP clients), but browsers will be blocked unless their origin is whitelisted.
|
||||
|
||||
## Complete Kubernetes Configuration Example
|
||||
|
||||
Here's your updated k8s/configmap.yaml with CORS:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: notifier-config
|
||||
labels:
|
||||
app: notifier
|
||||
data:
|
||||
config.yaml: |
|
||||
server:
|
||||
grpc_port: 50051
|
||||
rest_port: 8080
|
||||
host: "0.0.0.0"
|
||||
mode: "both"
|
||||
|
||||
queue:
|
||||
type: "local"
|
||||
max_size: 10000
|
||||
worker_count: 10
|
||||
retry_attempts: 3
|
||||
retry_backoff: "exponential"
|
||||
local:
|
||||
buffer_size: 1000
|
||||
persist_to_disk: false
|
||||
|
||||
notifiers:
|
||||
stdout: true
|
||||
|
||||
logging:
|
||||
level: "info"
|
||||
format: "json"
|
||||
output_path: "stdout"
|
||||
|
||||
metrics:
|
||||
enabled: true
|
||||
port: 9090
|
||||
path: "/metrics"
|
||||
prometheus_enabled: true
|
||||
|
||||
health_check:
|
||||
enabled: true
|
||||
port: 8081
|
||||
path: "/health"
|
||||
interval: 30
|
||||
|
||||
# CORS configuration for REST API
|
||||
# Only needed if web browsers will access the REST API
|
||||
cors:
|
||||
# Add your frontend domains here
|
||||
# For backend-only services, leave empty
|
||||
allowed_origins: []
|
||||
|
||||
# Or if you have a web frontend:
|
||||
# allowed_origins:
|
||||
# - "https://app.example.com"
|
||||
|
||||
allowed_methods:
|
||||
- "GET"
|
||||
- "POST"
|
||||
- "OPTIONS"
|
||||
- "DELETE"
|
||||
allowed_headers:
|
||||
- "Content-Type"
|
||||
- "Authorization"
|
||||
allow_credentials: false
|
||||
max_age: 3600
|
||||
```
|
||||
|
||||
## Environment-Specific Configuration
|
||||
|
||||
You can also use environment variables for different environments:
|
||||
|
||||
### Development ConfigMap
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: notifier-config-dev
|
||||
namespace: development
|
||||
data:
|
||||
NOTIFIER_CORS_ALLOWED_ORIGINS: "http://localhost:3000,http://localhost:8080"
|
||||
NOTIFIER_CORS_ALLOW_CREDENTIALS: "false"
|
||||
```
|
||||
|
||||
### Production ConfigMap
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: notifier-config-prod
|
||||
namespace: production
|
||||
data:
|
||||
NOTIFIER_CORS_ALLOWED_ORIGINS: "https://app.example.com,https://dashboard.example.com"
|
||||
NOTIFIER_CORS_ALLOW_CREDENTIALS: "true"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "CORS error" in Browser Console
|
||||
|
||||
**Symptom:**
|
||||
```
|
||||
Access to fetch at 'https://notifier.example.com/api/v1/notifications'
|
||||
from origin 'https://app.example.com' has been blocked by CORS policy
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
Add `https://app.example.com` to `allowed_origins` in your ConfigMap.
|
||||
|
||||
### Backend Service Can't Connect
|
||||
|
||||
**Symptom:**
|
||||
```go
|
||||
// Service in cluster trying to connect
|
||||
conn, err := grpc.Dial("notifier-grpc:50051", grpc.WithInsecure())
|
||||
// Error: connection refused
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
This is NOT a CORS issue. Check:
|
||||
1. Service name is correct (`notifier-grpc`)
|
||||
2. Port is correct (`50051`)
|
||||
3. Service is running: `kubectl get pods -l app=notifier`
|
||||
4. Service endpoints: `kubectl get endpoints notifier-grpc`
|
||||
|
||||
### REST API Returns 200 but No CORS Headers
|
||||
|
||||
**Symptom:**
|
||||
Browser blocks the response even though the API returns 200 OK.
|
||||
|
||||
**Solution:**
|
||||
The origin is not in the whitelist. Check:
|
||||
1. Origin is exactly matching (including protocol and port)
|
||||
2. ConfigMap has been updated
|
||||
3. Pod has been restarted to pick up new config:
|
||||
```bash
|
||||
kubectl rollout restart deployment notifier
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### For Backend Services Only (Recommended)
|
||||
```yaml
|
||||
cors:
|
||||
allowed_origins: [] # Empty - no browser access needed
|
||||
```
|
||||
|
||||
### For Web Frontend + Backend Services
|
||||
```yaml
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "https://your-frontend-domain.com"
|
||||
allow_credentials: true # If using auth tokens
|
||||
```
|
||||
|
||||
### Applying Changes
|
||||
|
||||
After updating the ConfigMap:
|
||||
|
||||
```bash
|
||||
# Update ConfigMap
|
||||
kubectl apply -f k8s/configmap.yaml
|
||||
|
||||
# Restart pods to pick up new config
|
||||
kubectl rollout restart deployment notifier
|
||||
|
||||
# Verify pods are running
|
||||
kubectl get pods -l app=notifier
|
||||
|
||||
# Check logs for CORS configuration
|
||||
kubectl logs -l app=notifier | grep CORS
|
||||
```
|
||||
|
||||
You should see:
|
||||
```
|
||||
CORS enabled for origins: [https://app.example.com]
|
||||
```
|
||||
|
||||
Or if no origins configured:
|
||||
```
|
||||
CORS has no allowed origins configured - all cross-origin requests will be blocked
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
**For your use case (services in Kubernetes using gRPC):**
|
||||
|
||||
✅ **You don't need to configure CORS at all!**
|
||||
|
||||
CORS only applies to web browsers accessing the REST API. Your backend services communicating via gRPC are completely unaffected by CORS configuration.
|
||||
|
||||
**Only add CORS configuration if:**
|
||||
- You have a web frontend (React, Vue, Angular, etc.)
|
||||
- That web frontend calls the REST API (not gRPC)
|
||||
- The frontend is served from a different domain than the API
|
||||
|
||||
For purely backend service-to-service communication in Kubernetes, CORS is irrelevant.
|
||||
@@ -0,0 +1,607 @@
|
||||
# gRPC Connection Optimization Guide
|
||||
|
||||
## TL;DR
|
||||
|
||||
**Yes, you can and should use long-lived gRPC connections!**
|
||||
|
||||
- ✅ Single connection can handle thousands of concurrent requests
|
||||
- ✅ Authentication happens **per-request**, not per-connection
|
||||
- ✅ Connection reuse eliminates TCP/TLS handshake overhead
|
||||
- ✅ HTTP/2 multiplexing allows concurrent RPCs on one connection
|
||||
- ✅ Built-in keepalive prevents connection timeouts
|
||||
|
||||
## How gRPC Authentication Works
|
||||
|
||||
### Key Point: Auth is Per-Request, Not Per-Connection
|
||||
|
||||
Your current implementation authenticates **each RPC call**, not the connection itself. This means:
|
||||
|
||||
1. Client establishes a **long-lived connection** (once)
|
||||
2. Client sends **API key in metadata** with each request
|
||||
3. Server validates the key for **every RPC call**
|
||||
4. Connection stays open for multiple requests
|
||||
|
||||
```
|
||||
Connection Lifecycle:
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ TCP Connection (persistent) │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ TLS Handshake (once) │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ Request 1: Authorization: Bearer nk_abc... → Validated │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ Request 2: Authorization: Bearer nk_abc... → Validated │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ Request 3: Authorization: Bearer nk_abc... → Validated │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
│ ... (connection stays open) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Authentication Overhead Analysis
|
||||
|
||||
**Per-Connection (One-Time):**
|
||||
- TCP handshake: ~1-2ms (3-way handshake)
|
||||
- TLS handshake: ~5-10ms (certificate exchange, key agreement)
|
||||
- **Total: ~10-15ms once**
|
||||
|
||||
**Per-Request (Every Call):**
|
||||
- API key validation: ~0.1-1ms (in-memory lookup)
|
||||
- Rate limit check: ~0.1ms (in-memory counter)
|
||||
- **Total: ~0.2-1ms per request**
|
||||
|
||||
**With Connection Reuse:**
|
||||
- First request: 10-15ms (connection) + 1ms (auth) = **11-16ms**
|
||||
- Subsequent requests: **1ms** (only auth, no connection setup)
|
||||
|
||||
**Without Connection Reuse (reconnecting each time):**
|
||||
- Every request: 10-15ms (connection) + 1ms (auth) = **11-16ms**
|
||||
|
||||
**Savings: 10-15ms per request after the first one!**
|
||||
|
||||
## Recommended Client Pattern
|
||||
|
||||
### Basic Long-Lived Connection
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
pb "github.com/igodwin/notifier/api/grpc/pb"
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/credentials/insecure"
|
||||
"google.golang.org/grpc/metadata"
|
||||
)
|
||||
|
||||
type NotifierClient struct {
|
||||
conn *grpc.ClientConn
|
||||
client pb.NotifierServiceClient
|
||||
apiKey string
|
||||
}
|
||||
|
||||
func NewNotifierClient(address, apiKey string) (*NotifierClient, error) {
|
||||
// Establish long-lived connection
|
||||
conn, err := grpc.Dial(address,
|
||||
grpc.WithTransportCredentials(insecure.NewCredentials()),
|
||||
|
||||
// Connection pool settings
|
||||
grpc.WithDefaultCallOptions(
|
||||
grpc.MaxCallRecvMsgSize(4*1024*1024), // 4MB
|
||||
grpc.MaxCallSendMsgSize(4*1024*1024),
|
||||
),
|
||||
|
||||
// Keepalive settings
|
||||
grpc.WithKeepaliveParams(keepalive.ClientParameters{
|
||||
Time: 10 * time.Second, // Send keepalive ping every 10s
|
||||
Timeout: 3 * time.Second, // Wait 3s for ping ack
|
||||
PermitWithoutStream: true, // Allow pings when no streams
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &NotifierClient{
|
||||
conn: conn,
|
||||
client: pb.NewNotifierServiceClient(conn),
|
||||
apiKey: apiKey,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (nc *NotifierClient) SendNotification(ctx context.Context, req *pb.SendNotificationRequest) (*pb.SendNotificationResponse, error) {
|
||||
// Add API key to metadata for THIS request
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "authorization", "Bearer "+nc.apiKey)
|
||||
|
||||
// Make RPC call - reuses existing connection
|
||||
return nc.client.SendNotification(ctx, req)
|
||||
}
|
||||
|
||||
func (nc *NotifierClient) Close() error {
|
||||
return nc.conn.Close()
|
||||
}
|
||||
|
||||
func main() {
|
||||
// Create client with long-lived connection
|
||||
client, err := NewNotifierClient("notifier-grpc:50051", "nk_your_api_key")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer client.Close() // Close when application exits
|
||||
|
||||
// Reuse client for multiple requests
|
||||
for i := 0; i < 1000; i++ {
|
||||
ctx := context.Background()
|
||||
resp, err := client.SendNotification(ctx, &pb.SendNotificationRequest{
|
||||
Type: pb.NotificationType_NOTIFICATION_TYPE_EMAIL,
|
||||
Subject: "Test notification",
|
||||
Body: "This is a test",
|
||||
Recipients: []string{"user@example.com"},
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
log.Printf("Request %d failed: %v", i, err)
|
||||
continue
|
||||
}
|
||||
|
||||
log.Printf("Request %d succeeded: %s", i, resp.NotificationId)
|
||||
}
|
||||
|
||||
// Connection is closed when main() exits
|
||||
}
|
||||
```
|
||||
|
||||
### Advanced: Connection Pool for High Concurrency
|
||||
|
||||
For extremely high throughput, you can create multiple connections:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
|
||||
pb "github.com/igodwin/notifier/api/grpc/pb"
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/keepalive"
|
||||
)
|
||||
|
||||
type NotifierPool struct {
|
||||
connections []*grpc.ClientConn
|
||||
clients []pb.NotifierServiceClient
|
||||
apiKey string
|
||||
current uint32
|
||||
mu sync.Mutex
|
||||
}
|
||||
|
||||
func NewNotifierPool(address, apiKey string, poolSize int) (*NotifierPool, error) {
|
||||
pool := &NotifierPool{
|
||||
connections: make([]*grpc.ClientConn, poolSize),
|
||||
clients: make([]pb.NotifierServiceClient, poolSize),
|
||||
apiKey: apiKey,
|
||||
}
|
||||
|
||||
// Create multiple connections
|
||||
for i := 0; i < poolSize; i++ {
|
||||
conn, err := grpc.Dial(address,
|
||||
grpc.WithTransportCredentials(insecure.NewCredentials()),
|
||||
grpc.WithKeepaliveParams(keepalive.ClientParameters{
|
||||
Time: 10 * time.Second,
|
||||
Timeout: 3 * time.Second,
|
||||
PermitWithoutStream: true,
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
// Clean up any connections already created
|
||||
pool.Close()
|
||||
return nil, err
|
||||
}
|
||||
|
||||
pool.connections[i] = conn
|
||||
pool.clients[i] = pb.NewNotifierServiceClient(conn)
|
||||
}
|
||||
|
||||
return pool, nil
|
||||
}
|
||||
|
||||
func (np *NotifierPool) getClient() pb.NotifierServiceClient {
|
||||
// Round-robin connection selection
|
||||
np.mu.Lock()
|
||||
defer np.mu.Unlock()
|
||||
|
||||
idx := np.current % uint32(len(np.clients))
|
||||
np.current++
|
||||
return np.clients[idx]
|
||||
}
|
||||
|
||||
func (np *NotifierPool) SendNotification(ctx context.Context, req *pb.SendNotificationRequest) (*pb.SendNotificationResponse, error) {
|
||||
// Add API key to metadata
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "authorization", "Bearer "+np.apiKey)
|
||||
|
||||
// Get a client from pool (round-robin)
|
||||
client := np.getClient()
|
||||
|
||||
return client.SendNotification(ctx, req)
|
||||
}
|
||||
|
||||
func (np *NotifierPool) Close() error {
|
||||
var firstErr error
|
||||
for _, conn := range np.connections {
|
||||
if conn != nil {
|
||||
if err := conn.Close(); err != nil && firstErr == nil {
|
||||
firstErr = err
|
||||
}
|
||||
}
|
||||
}
|
||||
return firstErr
|
||||
}
|
||||
|
||||
// Usage
|
||||
func main() {
|
||||
// Create pool with 4 connections
|
||||
pool, err := NewNotifierPool("notifier-grpc:50051", "nk_your_api_key", 4)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
|
||||
// Use pool concurrently from multiple goroutines
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 1000; i++ {
|
||||
wg.Add(1)
|
||||
go func(id int) {
|
||||
defer wg.Done()
|
||||
|
||||
resp, err := pool.SendNotification(context.Background(), &pb.SendNotificationRequest{
|
||||
Type: pb.NotificationType_NOTIFICATION_TYPE_EMAIL,
|
||||
Subject: fmt.Sprintf("Notification %d", id),
|
||||
Body: "Test",
|
||||
Recipients: []string{"user@example.com"},
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
log.Printf("Request %d failed: %v", id, err)
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("Request %d succeeded: %s", id, resp.NotificationId)
|
||||
}(i)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
}
|
||||
```
|
||||
|
||||
## Keepalive Configuration
|
||||
|
||||
### Why Keepalive Matters
|
||||
|
||||
In Kubernetes, idle connections may be terminated by:
|
||||
- Load balancers (after 60-600 seconds)
|
||||
- Network proxies
|
||||
- Firewalls with connection tracking
|
||||
|
||||
**Solution:** Send periodic keepalive pings
|
||||
|
||||
### Client-Side Keepalive
|
||||
|
||||
```go
|
||||
grpc.WithKeepaliveParams(keepalive.ClientParameters{
|
||||
Time: 10 * time.Second, // Send ping every 10s of inactivity
|
||||
Timeout: 3 * time.Second, // Wait 3s for ping response
|
||||
PermitWithoutStream: true, // Send pings even when no active RPCs
|
||||
})
|
||||
```
|
||||
|
||||
### Server-Side Keepalive (Already Configured in Your Server)
|
||||
|
||||
Add to `cmd/server/main.go` in `startGRPCServer()`:
|
||||
|
||||
```go
|
||||
serverOpts = append(serverOpts,
|
||||
grpc.KeepaliveParams(keepalive.ServerParameters{
|
||||
MaxConnectionIdle: 15 * time.Minute, // Close idle connections after 15m
|
||||
MaxConnectionAge: 30 * time.Minute, // Force close after 30m
|
||||
MaxConnectionAgeGrace: 5 * time.Second, // Allow 5s for RPCs to complete
|
||||
Time: 5 * time.Second, // Send ping if idle for 5s
|
||||
Timeout: 1 * time.Second, // Wait 1s for ping response
|
||||
}),
|
||||
grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
|
||||
MinTime: 5 * time.Second, // Don't allow pings more often than 5s
|
||||
PermitWithoutStream: true, // Allow pings when no streams
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
## Performance Comparison
|
||||
|
||||
### Scenario 1: Short-Lived Connections (Creating new connection for each request)
|
||||
|
||||
```
|
||||
Request 1: 15ms (10ms connect + 5ms TLS + 1ms auth + 0.5ms RPC)
|
||||
Request 2: 15ms (10ms connect + 5ms TLS + 1ms auth + 0.5ms RPC)
|
||||
Request 3: 15ms (10ms connect + 5ms TLS + 1ms auth + 0.5ms RPC)
|
||||
...
|
||||
1000 requests: ~15,000ms (15 seconds)
|
||||
```
|
||||
|
||||
### Scenario 2: Long-Lived Connection (Recommended)
|
||||
|
||||
```
|
||||
Request 1: 15ms (10ms connect + 5ms TLS + 1ms auth + 0.5ms RPC)
|
||||
Request 2: 1.5ms (1ms auth + 0.5ms RPC)
|
||||
Request 3: 1.5ms (1ms auth + 0.5ms RPC)
|
||||
...
|
||||
1000 requests: ~1,515ms (1.5 seconds)
|
||||
```
|
||||
|
||||
**Performance Improvement: 10x faster! (15s → 1.5s)**
|
||||
|
||||
### Scenario 3: Connection Pool with 4 Connections
|
||||
|
||||
```
|
||||
First 4 requests: 15ms each (connection setup)
|
||||
Remaining 996: 1.5ms each (reuse connections)
|
||||
1000 requests: ~1,554ms (1.5 seconds)
|
||||
Handles concurrent load better
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Connection Lifecycle Management
|
||||
|
||||
```go
|
||||
// Application-scoped client (singleton)
|
||||
var (
|
||||
notifierClient *NotifierClient
|
||||
once sync.Once
|
||||
)
|
||||
|
||||
func GetNotifierClient() *NotifierClient {
|
||||
once.Do(func() {
|
||||
client, err := NewNotifierClient(
|
||||
os.Getenv("NOTIFIER_ADDRESS"),
|
||||
os.Getenv("NOTIFIER_API_KEY"),
|
||||
)
|
||||
if err != nil {
|
||||
log.Fatalf("Failed to create notifier client: %v", err)
|
||||
}
|
||||
notifierClient = client
|
||||
})
|
||||
return notifierClient
|
||||
}
|
||||
|
||||
// In main():
|
||||
func main() {
|
||||
client := GetNotifierClient()
|
||||
defer client.Close()
|
||||
|
||||
// ... run application ...
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Context with Timeout
|
||||
|
||||
Always use context with timeout to prevent hanging requests:
|
||||
|
||||
```go
|
||||
func (nc *NotifierClient) SendNotification(ctx context.Context, req *pb.SendNotificationRequest) (*pb.SendNotificationResponse, error) {
|
||||
// Set timeout for this request
|
||||
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// Add API key
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "authorization", "Bearer "+nc.apiKey)
|
||||
|
||||
return nc.client.SendNotification(ctx, req)
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Health Checks
|
||||
|
||||
Periodically verify the connection is healthy:
|
||||
|
||||
```go
|
||||
func (nc *NotifierClient) HealthCheck(ctx context.Context) error {
|
||||
ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// Add API key
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "authorization", "Bearer "+nc.apiKey)
|
||||
|
||||
_, err := nc.client.HealthCheck(ctx, &pb.HealthCheckRequest{})
|
||||
return err
|
||||
}
|
||||
|
||||
// In background goroutine:
|
||||
go func() {
|
||||
ticker := time.NewTicker(30 * time.Second)
|
||||
defer ticker.Stop()
|
||||
|
||||
for range ticker.C {
|
||||
if err := client.HealthCheck(context.Background()); err != nil {
|
||||
log.Printf("Health check failed: %v", err)
|
||||
// Consider reconnecting or alerting
|
||||
}
|
||||
}
|
||||
}()
|
||||
```
|
||||
|
||||
### 4. Graceful Reconnection
|
||||
|
||||
Handle connection failures gracefully:
|
||||
|
||||
```go
|
||||
type ResilientNotifierClient struct {
|
||||
address string
|
||||
apiKey string
|
||||
client *NotifierClient
|
||||
mu sync.RWMutex
|
||||
}
|
||||
|
||||
func (rnc *ResilientNotifierClient) SendNotification(ctx context.Context, req *pb.SendNotificationRequest) (*pb.SendNotificationResponse, error) {
|
||||
rnc.mu.RLock()
|
||||
client := rnc.client
|
||||
rnc.mu.RUnlock()
|
||||
|
||||
resp, err := client.SendNotification(ctx, req)
|
||||
if err != nil && isConnectionError(err) {
|
||||
// Try to reconnect
|
||||
log.Printf("Connection error, attempting reconnect: %v", err)
|
||||
if err := rnc.reconnect(); err != nil {
|
||||
return nil, fmt.Errorf("reconnection failed: %w", err)
|
||||
}
|
||||
|
||||
// Retry request with new connection
|
||||
rnc.mu.RLock()
|
||||
client = rnc.client
|
||||
rnc.mu.RUnlock()
|
||||
|
||||
return client.SendNotification(ctx, req)
|
||||
}
|
||||
|
||||
return resp, err
|
||||
}
|
||||
|
||||
func (rnc *ResilientNotifierClient) reconnect() error {
|
||||
rnc.mu.Lock()
|
||||
defer rnc.mu.Unlock()
|
||||
|
||||
// Close old connection
|
||||
if rnc.client != nil {
|
||||
rnc.client.Close()
|
||||
}
|
||||
|
||||
// Create new connection
|
||||
client, err := NewNotifierClient(rnc.address, rnc.apiKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
rnc.client = client
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
## Kubernetes Deployment Considerations
|
||||
|
||||
### 1. Service Configuration
|
||||
|
||||
Your services are already correctly configured for long-lived connections:
|
||||
|
||||
```yaml
|
||||
# k8s/service.yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: notifier-grpc
|
||||
spec:
|
||||
type: ClusterIP # ✅ Good: Stable internal endpoint
|
||||
ports:
|
||||
- port: 50051
|
||||
targetPort: grpc
|
||||
protocol: TCP
|
||||
```
|
||||
|
||||
### 2. Client Connection String
|
||||
|
||||
```go
|
||||
// Within same namespace
|
||||
client, _ := NewNotifierClient("notifier-grpc:50051", apiKey)
|
||||
|
||||
// From different namespace
|
||||
client, _ := NewNotifierClient("notifier-grpc.default.svc.cluster.local:50051", apiKey)
|
||||
```
|
||||
|
||||
### 3. Load Balancing
|
||||
|
||||
Kubernetes service provides **connection-level** load balancing. For better **request-level** load balancing with long-lived connections, consider:
|
||||
|
||||
**Option A: Client-Side Load Balancing**
|
||||
|
||||
```go
|
||||
import "google.golang.org/grpc/resolver"
|
||||
|
||||
conn, err := grpc.Dial(
|
||||
"dns:///notifier-grpc:50051", // DNS resolver
|
||||
grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`),
|
||||
grpc.WithTransportCredentials(insecure.NewCredentials()),
|
||||
)
|
||||
```
|
||||
|
||||
**Option B: Connection Pool** (shown earlier)
|
||||
|
||||
## Rate Limiting Considerations
|
||||
|
||||
### Your Current Implementation
|
||||
|
||||
The rate limiter checks on **every request** (in `grpc_middleware.go:46-50`):
|
||||
|
||||
```go
|
||||
allowed, err := m.store.CheckRateLimit(apiKey)
|
||||
if err != nil || !allowed {
|
||||
return nil, status.Error(codes.ResourceExhausted, "Rate limit exceeded")
|
||||
}
|
||||
```
|
||||
|
||||
### Impact with Long-Lived Connections
|
||||
|
||||
**No negative impact!** Rate limiting works the same:
|
||||
- Each RPC call is checked independently
|
||||
- Connection reuse doesn't bypass rate limits
|
||||
- Rate limit is per API key, not per connection
|
||||
|
||||
## Summary
|
||||
|
||||
### ✅ DO: Use Long-Lived Connections
|
||||
|
||||
```go
|
||||
// Create once at application startup
|
||||
client, _ := NewNotifierClient("notifier-grpc:50051", apiKey)
|
||||
defer client.Close()
|
||||
|
||||
// Reuse for all requests
|
||||
for {
|
||||
client.SendNotification(ctx, req)
|
||||
}
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- 10x faster (eliminates connection setup overhead)
|
||||
- Lower latency (1.5ms vs 15ms per request)
|
||||
- Fewer resources (one connection vs. thousands)
|
||||
- Better throughput (HTTP/2 multiplexing)
|
||||
- Automatic keepalive prevents timeouts
|
||||
|
||||
### ❌ DON'T: Create Connection Per Request
|
||||
|
||||
```go
|
||||
// BAD: Don't do this!
|
||||
for {
|
||||
client, _ := NewNotifierClient("notifier-grpc:50051", apiKey)
|
||||
client.SendNotification(ctx, req)
|
||||
client.Close()
|
||||
}
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Slow (15ms per request)
|
||||
- Wasteful (repeated TCP/TLS handshakes)
|
||||
- Resource-intensive (thousands of connections)
|
||||
|
||||
### Authentication Still Happens Per-Request
|
||||
|
||||
- API key sent in metadata with every RPC
|
||||
- Server validates on every call
|
||||
- No security trade-off
|
||||
- Just eliminates connection setup overhead
|
||||
|
||||
**You get both: Maximum performance AND full security!**
|
||||
@@ -0,0 +1,319 @@
|
||||
# Quick Start: Notifier Client Integration
|
||||
|
||||
**5-minute guide to integrating with the Notifier service**
|
||||
|
||||
## Essential Requirements
|
||||
|
||||
### 1. Connection: Use Long-Lived gRPC Connection ⚡
|
||||
|
||||
```go
|
||||
// ✅ DO THIS: Create once, reuse forever
|
||||
client, _ := NewNotifierClient("notifier-grpc:50051", apiKey)
|
||||
defer client.Close()
|
||||
|
||||
for i := 0; i < 10000; i++ {
|
||||
client.SendNotification(ctx, req) // Reuse connection
|
||||
}
|
||||
|
||||
// ❌ DON'T DO THIS: Creating connection per request
|
||||
for i := 0; i < 10000; i++ {
|
||||
client, _ := NewNotifierClient("notifier-grpc:50051", apiKey)
|
||||
client.SendNotification(ctx, req)
|
||||
client.Close() // SLOW: 10x slower!
|
||||
}
|
||||
```
|
||||
|
||||
**Why?** Connection reuse is 10x faster (1.5ms vs 15ms per request)
|
||||
|
||||
### 2. Authentication: Send API Key with Every Request 🔐
|
||||
|
||||
```go
|
||||
import "google.golang.org/grpc/metadata"
|
||||
|
||||
// Add API key to metadata for each request
|
||||
ctx = metadata.AppendToOutgoingContext(ctx,
|
||||
"authorization", "Bearer "+apiKey)
|
||||
|
||||
resp, err := client.SendNotification(ctx, req)
|
||||
```
|
||||
|
||||
**Two accepted formats:**
|
||||
- `authorization: Bearer nk_your_api_key` (preferred)
|
||||
- `x-api-key: nk_your_api_key`
|
||||
|
||||
**API Key Format:** `nk_` + 64 hex characters (e.g., `nk_a1b2c3d4...`)
|
||||
|
||||
### 3. Credentials: Load from Environment 🔒
|
||||
|
||||
```go
|
||||
// ✅ DO THIS
|
||||
apiKey := os.Getenv("NOTIFIER_API_KEY")
|
||||
|
||||
// ❌ DON'T DO THIS
|
||||
apiKey := "nk_abc123..." // NEVER hardcode!
|
||||
```
|
||||
|
||||
**Kubernetes Secret:**
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: notifier-credentials
|
||||
stringData:
|
||||
api-key: nk_your_actual_key_here
|
||||
---
|
||||
# In your deployment:
|
||||
env:
|
||||
- name: NOTIFIER_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: notifier-credentials
|
||||
key: api-key
|
||||
```
|
||||
|
||||
### 4. Keepalive: Prevent Connection Timeouts ⏰
|
||||
|
||||
```go
|
||||
import "google.golang.org/grpc/keepalive"
|
||||
|
||||
conn, err := grpc.Dial(address,
|
||||
grpc.WithKeepaliveParams(keepalive.ClientParameters{
|
||||
Time: 10 * time.Second, // Ping every 10s
|
||||
Timeout: 3 * time.Second, // Wait 3s for response
|
||||
PermitWithoutStream: true, // Ping when idle
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
### 5. Error Handling: Retry Only Specific Errors 🔄
|
||||
|
||||
```go
|
||||
err := client.SendNotification(ctx, req)
|
||||
|
||||
switch status.Code(err) {
|
||||
case codes.ResourceExhausted:
|
||||
// Rate limit exceeded - RETRY with backoff
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
// Try again...
|
||||
|
||||
case codes.Unauthenticated:
|
||||
// Invalid API key - DON'T RETRY, log error
|
||||
log.Errorf("Authentication failed: %v", err)
|
||||
|
||||
case codes.InvalidArgument:
|
||||
// Bad request - DON'T RETRY, fix request
|
||||
log.Errorf("Invalid request: %v", err)
|
||||
|
||||
case codes.Unavailable:
|
||||
// Service down - RETRY with backoff
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
// Try again...
|
||||
}
|
||||
```
|
||||
|
||||
**Retry Strategy:**
|
||||
- Exponential backoff: 100ms → 200ms → 400ms → 800ms
|
||||
- Max 3-5 attempts
|
||||
- Only retry: `ResourceExhausted`, `Unavailable`, `DeadlineExceeded`
|
||||
|
||||
### 6. Context Timeout: Always Set Timeout ⏱️
|
||||
|
||||
```go
|
||||
// ✅ DO THIS: Set timeout for each request
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
resp, err := client.SendNotification(ctx, req)
|
||||
|
||||
// ❌ DON'T DO THIS: No timeout
|
||||
ctx := context.Background()
|
||||
resp, err := client.SendNotification(ctx, req) // Could hang forever!
|
||||
```
|
||||
|
||||
## Complete Minimal Example
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
pb "github.com/igodwin/notifier/api/grpc/pb"
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/credentials/insecure"
|
||||
"google.golang.org/grpc/keepalive"
|
||||
"google.golang.org/grpc/metadata"
|
||||
)
|
||||
|
||||
type NotifierClient struct {
|
||||
conn *grpc.ClientConn
|
||||
client pb.NotifierServiceClient
|
||||
apiKey string
|
||||
}
|
||||
|
||||
func NewNotifierClient(address, apiKey string) (*NotifierClient, error) {
|
||||
// Create long-lived connection
|
||||
conn, err := grpc.Dial(address,
|
||||
grpc.WithTransportCredentials(insecure.NewCredentials()),
|
||||
grpc.WithKeepaliveParams(keepalive.ClientParameters{
|
||||
Time: 10 * time.Second,
|
||||
Timeout: 3 * time.Second,
|
||||
PermitWithoutStream: true,
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &NotifierClient{
|
||||
conn: conn,
|
||||
client: pb.NewNotifierServiceClient(conn),
|
||||
apiKey: apiKey,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (nc *NotifierClient) SendNotification(ctx context.Context, req *pb.SendNotificationRequest) (*pb.SendNotificationResponse, error) {
|
||||
// Add timeout
|
||||
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// Add API key to metadata
|
||||
ctx = metadata.AppendToOutgoingContext(ctx, "authorization", "Bearer "+nc.apiKey)
|
||||
|
||||
// Make RPC call
|
||||
return nc.client.SendNotification(ctx, req)
|
||||
}
|
||||
|
||||
func (nc *NotifierClient) Close() error {
|
||||
return nc.conn.Close()
|
||||
}
|
||||
|
||||
func main() {
|
||||
// Load credentials
|
||||
address := os.Getenv("NOTIFIER_ADDRESS")
|
||||
apiKey := os.Getenv("NOTIFIER_API_KEY")
|
||||
|
||||
if address == "" {
|
||||
address = "notifier-grpc:50051"
|
||||
}
|
||||
if apiKey == "" {
|
||||
log.Fatal("NOTIFIER_API_KEY environment variable required")
|
||||
}
|
||||
|
||||
// Create client (once)
|
||||
client, err := NewNotifierClient(address, apiKey)
|
||||
if err != nil {
|
||||
log.Fatalf("Failed to create client: %v", err)
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
// Send notification
|
||||
resp, err := client.SendNotification(context.Background(), &pb.SendNotificationRequest{
|
||||
Type: pb.NotificationType_NOTIFICATION_TYPE_EMAIL,
|
||||
Subject: "Test Notification",
|
||||
Body: "This is a test from the notifier client",
|
||||
Recipients: []string{"user@example.com"},
|
||||
Priority: pb.Priority_PRIORITY_NORMAL,
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
log.Fatalf("Failed to send notification: %v", err)
|
||||
}
|
||||
|
||||
log.Printf("Notification sent successfully: %s", resp.NotificationId)
|
||||
}
|
||||
```
|
||||
|
||||
## Service Connection Details
|
||||
|
||||
**Address:**
|
||||
- Same namespace: `notifier-grpc:50051`
|
||||
- Different namespace: `notifier-grpc.default.svc.cluster.local:50051`
|
||||
|
||||
**Rate Limit:** 100 requests/minute (default, configurable per key)
|
||||
|
||||
**Available Methods:**
|
||||
- `SendNotification` - Send single notification
|
||||
- `SendBatchNotifications` - Send multiple notifications
|
||||
- `GetNotification` - Get notification status
|
||||
- `ListNotifications` - List recent notifications
|
||||
- `CancelNotification` - Cancel pending notification
|
||||
- `RetryNotification` - Retry failed notification
|
||||
- `GetStats` - Get service statistics
|
||||
- `GetNotifiers` - List available notifiers
|
||||
- `HealthCheck` - Check service health
|
||||
|
||||
**Notification Types:**
|
||||
- `NOTIFICATION_TYPE_EMAIL` - Email via SMTP
|
||||
- `NOTIFICATION_TYPE_SLACK` - Slack webhook
|
||||
- `NOTIFICATION_TYPE_NTFY` - Ntfy.sh push notifications
|
||||
- `NOTIFICATION_TYPE_STDOUT` - Console output (dev only)
|
||||
|
||||
## Security Checklist
|
||||
|
||||
- [ ] API key loaded from environment variable
|
||||
- [ ] No hardcoded credentials in source code
|
||||
- [ ] API key never logged in full (mask it: `nk_abc...xyz`)
|
||||
- [ ] Kubernetes Secret created for API key
|
||||
- [ ] Deployment configured to load secret as env var
|
||||
- [ ] `.gitignore` excludes any credential files
|
||||
|
||||
## Testing Your Integration
|
||||
|
||||
```bash
|
||||
# Set environment variables
|
||||
export NOTIFIER_ADDRESS="notifier-grpc:50051"
|
||||
export NOTIFIER_API_KEY="nk_your_api_key_here"
|
||||
|
||||
# Run your application
|
||||
go run main.go
|
||||
|
||||
# Check logs for:
|
||||
# ✅ "Notification sent successfully: <id>"
|
||||
# ❌ "Authentication failed" - check API key
|
||||
# ❌ "Rate limit exceeded" - slow down requests
|
||||
```
|
||||
|
||||
## Common Issues
|
||||
|
||||
**"Unauthenticated" error:**
|
||||
- Check API key format starts with `nk_`
|
||||
- Verify key is 66 characters total (nk_ + 64 hex)
|
||||
- Confirm key is active in notifier service
|
||||
- Check you're adding it to metadata correctly
|
||||
|
||||
**"Rate limit exceeded" error:**
|
||||
- Implement exponential backoff retry
|
||||
- Consider batching notifications
|
||||
- Request higher rate limit for your key
|
||||
|
||||
**Connection timeouts:**
|
||||
- Add keepalive configuration
|
||||
- Check network connectivity to `notifier-grpc:50051`
|
||||
- Verify notifier service is running: `kubectl get pods -l app=notifier`
|
||||
|
||||
**Slow performance:**
|
||||
- Confirm you're reusing connection (not creating per request)
|
||||
- Check keepalive is configured
|
||||
- Verify you're using connection pooling for high concurrency
|
||||
|
||||
## Next Steps
|
||||
|
||||
For more details, see:
|
||||
- `CLIENT_INTEGRATION_PROMPT.md` - Complete prompt for AI assistants
|
||||
- `GRPC_CONNECTION_OPTIMIZATION.md` - Deep dive on performance
|
||||
- `CLIENT_RECOMMENDATIONS.md` - Advanced patterns and best practices
|
||||
- `AUTH.md` - Authentication system details
|
||||
|
||||
## Getting Your API Key
|
||||
|
||||
Contact your Notifier service administrator to:
|
||||
1. Create an API key for your application
|
||||
2. Set appropriate rate limits
|
||||
3. Assign required roles for notifier access
|
||||
4. Get the key in format: `nk_<64-hex-characters>`
|
||||
|
||||
Store it securely and never commit it to version control!
|
||||
Reference in New Issue
Block a user