Backend Engineer _ AI Gateway- Bilingual (Korean/ English)

Job Description: We are building an AI Gateway — a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

What we are looking for: Async Python & Network Programming (3 + yrs) – Strong hands‐on experience with asyncio, aiohttp/anyio, low‐level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection‐pooling, keep‐alive, and zero‐copy buffering. Streaming & Large‐Payload Handling – Ability to manage chunked transfer, bidirectional streaming, back‐pressure and high‐throughput data paths for real‐time LLM inference. Protocol Knowledge – Working familiarity with HTTP/2 & HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers. FastAPI‐Based Backend (3 + yrs) – Design, develop and test production‐grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs. Containerization & Kubernetes – Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service‐mesh (e.g., Istio/Envoy) basics. Async Messaging & Event‐Driven Design – Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back‐off logic. PostgreSQL & Vault – Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management. Cloud Networking & Security – L4/L7 load balancing, TLS termination, reverse‐proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let's Encrypt).IaC & CI/CD – Terraform modules for multi‐env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue‐green deployments.

Qualifications: Enterprise Security Gateways – Integration with CASB/DLP/WSS or similar edge‐security platforms; centralized logging & audit trails. IAM & SSO – Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning. AI/LLM Pipelines – Building prompt‐engineering workflows, PII anonymisation, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost. API‐Gateway Operations – Custom webhook/REST integrations, circuit‐breaker patterns, rate‐limiting, auto‐retry and health‐checking. Observability & FinOps – End‐to‐end monitoring with Datadog, Prometheus/Grafana or Azure Monitor; proactive cost‐optimisation and 24/7 incident response (SLA/SLO).Bilingual Korean Highly Prefer

Additional Information: All your information will be kept confidential according to EEO guidelines.***NO C2C****Summary

Type: Contract

Function: Information Technology

Experience level: Mid-Senior Level

Industry: Information Technology And Services