rakutentech/sorify

Sorify is an ambitious, well-structured self-hosted testing platform. It combines a Laravel 13 backend, Vue and Inertia interface, Playwright execution, scheduling, retries, screenshots, notifications, role-based suite access, MCP tools, and a Chrome recording extension. The implementation supports the README’s main claims rather than presenting unfinished placeholders. Test execution includes configurable browsers, cookies, variables, proxies, cancellation, history retention, and structured results. Background queues and a dedicated scheduler make sense for this workload, while the Docker setup packages MySQL, FrankenPHP, Chromium, workers, and migrations into a practical deployment. The repository also contains a meaningful PHPUnit suite covering permissions, MCP operations, schedules, authentication, variables, cookies, duplication, and code history. The Apache 2.0 license is clear, with the extension’s separate BSD-based terms documented.
The most important improvement is stronger isolation for generated Playwright code. Tests currently run through eval in a Node process inside the application’s worker environment. A regular-expression validator blocks several obvious APIs, but filtering source text is not a dependable security boundary for JavaScript. A compromised or untrusted test could potentially reach the filesystem, application secrets, internal services, or unrestricted network destinations. Each run should execute in a disposable container or sandbox with no application credentials, a read-only filesystem, restricted egress, and strict CPU, memory, and time limits.
The authentication and release practices could also mature alongside the feature set. MCP clients store the user’s account password in ~/.sorify and send Basic authentication, while scoped, revocable API tokens would reduce exposure. Webhook secrets are placed in URLs and stored directly, so header-based tokens and hashed storage would be safer. CI successfully builds the frontend and runs PHPUnit, but its “Test and Lint” workflow does not actually run a linter, Go tests, or browser and extension tests. Adding those checks, plus a security policy, contribution guide, upgrade and backup documentation, and production checks that reject default credentials, would make this promising project much easier to trust and operate.