Evaluation
We develop evaluation methods and tools to continuously track and evaluate risks in frontier models.
Fangcun AI is an AI safety company jointly incubated by Tsinghua University's College of AI and Institute for Interdisciplinary Information Sciences.
We evaluate frontier risks in model capabilities and behavior. We provide safeguards to mitigate risks in AI systems and contribute to AI governance.
We develop evaluation methods and tools to continuously track and evaluate risks in frontier models.
We use our own models, observation platform, and protection tools to secure model calls and agent deployments.
We contribute to AI and agent safety policy and international dialogue, drawing on evidence from our evaluations.
A runtime security platform for AI agents, with monitoring, risk analysis, and activity tracing.
Direct safety assessment
Step-by-step safety reasoning
Safety assessment across input modalities
National Technical Committee 260 on Cybersecurity
WG8 / WG9 / SWG-ETS
Artificial Intelligence Industry Alliance
China AI Development and Safety Research Network
Tsinghua AI Safety Hub
We publish research methods and findings on model risks, safeguards, and trustworthy AI applications.
Our research examines loss of control, deception and harmful manipulation in frontier AI models, as well as safety methods for models and agents.
We develop closed-source tests to maintain evaluation quality and independence. This also helps address reliability challenges when models recognize they are being evaluated.
The evaluation process matters as much as the results. We examine what happens during each evaluation and acknowledge its limitations.
Each report identifies whether the evaluation was commissioned or independently initiated. Evaluated organizations may verify factual details but cannot change the conclusions.
Fangcun Observer
Correlate agent tasks, tool calls, and risk events to monitor activity and analyze risks. Use verifiable execution records to trace behavior and support incident response.
Request early accessCorrelate tasks with execution records
Review file, command, and network events
Apply identity-based access policies
Check inputs and responses in connected workflows
Automate scans and link results to content versions
Assisted analysis; you decide whether to follow suggestions
Review token usage and calculated costs
Daily, weekly, and monthly runtime summaries
Browse optional extension resources
Direct safety assessment
Step-by-step safety reasoning
Safety assessment across input modalities
A security scanner for Skills and MCPs. It combines static analysis and honeypot execution, using runtime evidence to identify and explain risks.
Our assessment findings inform standards development in China and international dialogue on frontier AI safety.
National Technical Committee 260 on Cybersecurity
WG8 / WG9 / SWG-ETS
Artificial Intelligence Industry Alliance
China AI Development and Safety Research Network
Tsinghua AI Safety Hub
Xiaojian Li and Wei Xu participated.
Wei Xu participated.
Wei Xu and Xiaojian Li contributed to the consensus on risk assessment, trustworthy development, and post-deployment control.
A hub for collaboration on AI safety and risk research, supporting research residencies, early-career development and international exchange.
We disclose funding and interests relevant to our evaluations and use conflict-of-interest reviews and recusal procedures to maintain the independence of our evaluations.
For safety issues identified in our evaluations, we work with developers on remediation and publish research findings while managing the risk of misuse.
Fangcun AI is an AI-native security company, jointly incubated by Tsinghua University's College of AI and the Institute for Interdisciplinary Information Sciences.
Guided by our published independence and conflict-of-interest policy, we assess frontier model risks and train models for safety oversight. We also contribute to the AI safety community, open-source projects, and governance.
Assessments identify risks. Our products build safeguards into agent operations, and our governance work uses evaluation evidence to inform rules. These areas of work support one another.
Fangcun AI (Xiong'an) Technology Co., Ltd. (方寸跃迁(雄安)科技有限公司, referred to as “Fangcun AI” or “we”) values and protects your personal information. This Policy explains how we collect, use, store, entrust the processing of, provide and protect your personal information when you visit the Fangcun AI website, contact us, or use products and services that expressly link to this Policy. It also explains how you can exercise your related rights.
We process personal information in accordance with applicable laws and regulations, including the Personal Information Protection Law of the People's Republic of China, the Cybersecurity Law of the People's Republic of China, the Data Security Law of the People's Republic of China, and the Regulations on Network Data Security Management.
Except for specific products or services that have a separate privacy policy, this Privacy Policy applies to all Fangcun AI online pages, platforms and services that refer or link to it. It explains how we collect, use, disclose, process and protect personal information that you authorize us to receive or that we collect when providing products or services on our website and through the applications we offer.
If a product or service has a separate privacy policy, that policy takes precedence. This Privacy Policy applies to matters not covered by that product's or service's separate policy. How a particular product or service collects and processes your personal information may also vary depending on factors such as your operating system, service version or location. Please refer to the separate privacy policy for details.
To provide our website, products and services, we process the personal information necessary for the features you actually choose to use. We collect and use information only for specific, explicit and lawful purposes. Listing a category of information in this Policy does not mean we collect it in every situation. You may choose not to provide information that is not necessary for a particular feature; however, this may prevent us from fulfilling your request or providing that feature.
Depending on the features you use or how you interact with us, you may provide the following information:
When you visit our website or use an online service, our systems may automatically generate the following information:
Where permitted by law, we may obtain information about you from the following sources:
We require information providers to explain the source of the personal information and confirm that they have an appropriate lawful basis. We use the information we receive only for the purposes described in this Policy and to the extent necessary.
We may use the information described above for the following purposes:
We do not use content you submit or authorize us to process, or fault information you voluntarily submit, to train general-purpose models or detection models unless we expressly inform you separately and obtain the necessary authorization.
Some Fangcun AI products analyze the operational status and security events of AI systems or agents. The relevant data may be processed in the customer's local environment or in a designated environment, depending on the deployment method chosen by the customer and the contractual terms.
If the relevant data remains entirely in the customer's local environment and is not transmitted to us, we do not obtain that data. When a customer enables cloud features or remote support, or a user voluntarily submits fault information, diagnostic logs or other materials, we process the information received only within the scope of the relevant features, customer instructions and contractual terms.
Fault logs, screenshots, attachments or operational context may contain personal information, business data or third-party information. Before submitting them, please review the content and remove passwords, keys, access tokens and other sensitive information unrelated to troubleshooting.
When a standalone Observer application, client or cloud service is formally made available, we will provide a separate product privacy policy or specific notice based on its actual deployment methods, data scope and controls.
Depending on the circumstances, we may process personal information on one or more of the following grounds:
If there is a material change in the purpose or method of processing, or in the categories of personal information processed, we will provide a new notice as required by law. Where consent or separate consent is required, we will obtain it separately.
We generally do not ask you through our website to provide sensitive personal information such as biometric data, medical and health information, financial account information, precise location history or information about specific identities. However, content, logs, screenshots, attachments or résumés you submit may contain sensitive personal information.
If a feature does require the processing of sensitive personal information, we will explain why it is necessary, the specific purpose and method of processing, and its impact on individual rights and interests. We will obtain separate consent or complete other necessary procedures as required by law. Please do not submit sensitive personal information unrelated to your purpose of use.
We may use cookies, local storage or similar technologies to maintain website and account sessions, record necessary settings, protect security, troubleshoot problems, and understand the basic operation of our website and services.
Disabling strictly necessary cookies may prevent some website or service features from working properly. For non-essential statistics, analytics or marketing cookies, we will provide appropriate notice and choices to the extent required by applicable law. You can also manage or delete cookies through your browser settings.
If we use new third-party statistics, advertising or cross-site tracking tools in the future, we will update the relevant notices before enabling them and provide ways to refuse or withdraw as required by law.
To provide and maintain our website and services, we may entrust infrastructure, cloud computing, cybersecurity, email communications, technical support or payment service providers with processing necessary personal information. Our contracts specify the purpose, duration and method of processing, the categories of information, safeguards, and each party's responsibilities. We also supervise the entrusted parties' processing activities.
Based on the services currently offered through our website, the main third parties that may be involved include:
| Third party or service category | Processing context | Information that may be involved |
|---|---|---|
| Infrastructure service providers such as Alibaba Cloud | Website, servers, storage and security protection | Website access logs and information needed to operate accounts and services |
| Payment service providers such as Alipay and WeChat Pay (only when the relevant paid features are enabled) | Payments, refunds and reconciliation | Order number, amount and payment status; payment account information is processed by the relevant payment service provider under its own rules |
When we integrate a new third-party service that processes personal information, we will update this Policy, our list of information shared with third parties, or the relevant product page.
We do not sell your personal information. We do not provide it to other personal information handlers except in the following circumstances:
As a general rule, we do not publicly disclose your personal information. Where disclosure is necessary, we will inform you of its purpose and the categories of information involved and obtain separate consent as required by law, unless laws or regulations provide otherwise.
If you use our enterprise products through an account, device or environment provided by your organization, its administrators may access or manage accounts, configurations, operational records and audit information within their administrative permissions. Enterprise customers are responsible for determining lawful processing purposes and scope, notifying the individuals concerned, obtaining necessary authorization, and setting appropriate permissions and retention periods in accordance with the law.
When an enterprise customer entrusts us with processing personal information, we process it in accordance with the customer's lawful instructions and our contract, without independently changing the purpose or method of processing. For this data, you can generally first submit rights requests to your organization or the relevant enterprise customer. We will assist the enterprise customer in handling such requests as required by our contract and the law.
We retain personal information for the shortest period necessary to achieve the processing purpose. When the retention period expires or the information is no longer needed for that purpose, we delete or anonymize it in accordance with the law. If laws or regulations require continued retention, or deletion is technically difficult, we stop all processing other than storage and necessary security measures.
We generally determine retention periods using the following criteria:
| Information category | Typical retention period or how it is determined |
|---|---|
| Network and security logs | Retained as required by cybersecurity laws and regulations, generally for at least six months; deleted or anonymized once no longer necessary |
| Business inquiries, demo requests and correspondence | Retained while handling inquiries and maintaining business contact; if no ongoing business relationship is established, generally no more than three years from the last substantive communication |
| Account information | Retained while the account exists and services are provided; after account closure or service termination, deleted or anonymized once the period required for settlement, security audits and legal obligations has ended |
| User-submitted content and service records | Retained according to the specific product features, user settings, contractual terms or specific privacy notices |
| Voluntarily submitted fault and diagnostic information | Retained until the issue is resolved and the necessary review period ends, generally no longer than six months after the issue is closed, except where security incidents, disputes or legal obligations are involved |
| Recruitment materials | Generally retained for six months after the recruitment process ends; if we wish to include you in our talent pool, we will obtain separate consent and explain the retention period |
| Order, transaction and invoicing records | Retained for the periods required by applicable tax, accounting, e-commerce and other laws |
As a general rule, we store personal information collected and generated through our operations in the People's Republic of China within the People's Republic of China. If it is necessary to provide personal information outside China, we will inform you of the overseas recipient, the purpose and method of processing, the categories of information and how to exercise your rights; obtain separate consent; and complete a data export security assessment, personal information protection certification, standard contract or other applicable procedures as required by law.
We take reasonable technical and organizational measures appropriate to the types of personal information, processing circumstances and risks. These include access controls, authentication, encryption or de-identification, security audits, backups, permission management, personnel training and incident response, to prevent unauthorized access and the disclosure, alteration, loss or misuse of personal information.
Absolute security cannot be guaranteed on the internet. If personal information is or may be disclosed, altered or lost, we will take remedial measures in accordance with the law. Where required by law, we will report the incident to the relevant authorities and notify affected individuals of the incident, its possible impact, and measures taken or recommended.
To the extent provided by applicable law, you have the right to be informed about and make decisions concerning the processing of your personal information, and to restrict or refuse processing. You may also request to:
You can submit a request through the email address listed in the “Contact us” section of this Policy. To protect your personal information, we may ask for the information necessary to verify your identity. Once your identity is verified, we will handle your request promptly in accordance with the law. If we cannot fulfill your request, we will explain why. We may lawfully limit or refuse requests that are unreasonably repetitive, exceed a reasonable scope, require excessive technical effort, may harm others' lawful rights and interests, or do not require a response under the law.
Unless a deceased person made other arrangements during their lifetime, their close relatives may, for their own lawful and legitimate interests, exercise rights such as access, copying, correction and deletion regarding the deceased person's relevant personal information, in accordance with the law.
Withdrawing consent does not affect processing already carried out on the basis of that consent before its withdrawal. For information necessary to perform a contract, fulfill legal obligations or maintain security, withdrawing consent may not prevent us from continuing necessary processing on another lawful basis.
If we send you information about products, services or events, we will obtain the necessary authorization to the extent required by applicable law and provide a convenient way to unsubscribe or opt out.
We do not currently make decisions that significantly affect individuals' rights and interests solely through automated decision-making. If we undertake such processing in the future, we will ensure transparency, fairness and impartiality as required by law and provide explanations, the right to refuse decisions made solely through automated decision-making, or other applicable safeguards for your rights.
Our website and enterprise products are primarily intended for businesses, professionals and developers, and are not specifically directed at minors under 14. We do not knowingly solicit personal information from minors under 14.
If a product or service does need to process the personal information of minors under 14, we will establish specific personal information processing rules and obtain the consent of their parents or other guardians as required by law. If you discover that a minor has provided personal information to us without a guardian's consent, please contact us promptly. We will verify and address the matter in accordance with the law.
We may update this Policy to reflect business changes, product adjustments or legal and regulatory requirements. The updated Policy will be published prominently on our website and will show its last updated date.
If a change may significantly affect your personal information rights and interests, we will provide prominent notice through website prompts, in-service notifications, email or other appropriate means. For new processing activities that require consent by law, we will complete the applicable procedures separately rather than treating your continued use of the services as consent.
If you have any questions, comments, complaints or suggestions about this Policy, our personal information processing activities or rights requests, please contact us at the email address above. We will verify the relevant circumstances and handle the matter promptly in accordance with the law.
Frontier AI is becoming more capable at a rapid pace. Whether these systems are safe is a question the public, policymakers, and the industry all need answered credibly. Fangcun AI aims to contribute as an independent third party, providing objective and reliable safety evaluations that help developers identify and address problems, and that give the public and policymakers a sound basis for their decisions.
Independence is what allows a third-party evaluation to earn trust. Rigorous methods lead to credible conclusions only when others can be confident that those conclusions are not influenced by funding, equity, or personal relationships. This policy therefore sets clear boundaries on funding, paid work, equity, and personal relationships, and commits us to full disclosure in every published report.
This policy is designed to meet condition 2.3 of AEF-1, the AI Evaluator Forum's minimum operating conditions for independent third-party AI evaluations. It also supports conditions 2.1 (no contingent compensation), 2.2 (no organizational control), 2.4 (conflict-of-interest disclosure), and 2.5 (recusal). AEF-1 as used in this policy means Version 1, updated 4 December 2025.
This policy applies to every evaluation we conduct of a named frontier AI company's systems, and to everyone who takes part in one, including employees, interns, contractors, advisors, and members of the board of directors.
The following are outside its scope: research on general evaluation methods, reports that do not assess a specific company, and evaluations of AI applications built by companies in other industries.
Personnel must report potential conflicts when they join, every six months thereafter, whenever a new conflict arises, and before taking part in any evaluation of a named company. As a guiding principle, any relationship that, if made public, could give others reasonable grounds to question the impartiality of an evaluation should be reported.
| Level | Relationship with the evaluated company | Required action |
|---|---|---|
| Recusal | Current employment, contracting, or paid work there, or seeking it; direct equity or debt holdings, including through immediate family; immediate family working there; gifts from its staff beyond ordinary courtesy in the past six months | Recused from the evaluation |
| Disclosure | Any recusal-level relationship with a direct competitor; employment at the evaluated company within the past 24 months | Disclosed in the report; the person's work is checked by an unconflicted colleague |
| Case by case | Close relationship with an employee; past research collaboration; living with an employee of the evaluated company | Reviewed by the evaluation lead and documented; the information is used only for internal recusal and review and is not disclosed externally except in generalized form |
Holdings through index or mutual funds, and support roles carried out under qualified supervision, do not require recusal.
Every published evaluation, and every non-public report we deliver, must state whether any of the following apply, with details where they do:
Potential conflicts and suspected violations are reported to the operations lead or to the person selecting the evaluation team, who refers them to the leadership team for review. If the CEO's impartiality is in question, the board of directors reviews the case instead. If a director has a personal interest in a case, that director steps back and the remaining disinterested directors review it.
While a case is under review, the person concerned steps back from the evaluation. A violation can lead to removal from the evaluation, a correction to the published report, and disciplinary action. Disclosures and review decisions are recorded and kept on file. A person subject to a decision may appeal to the leadership team. Personnel who report conflicts or violations in good faith are protected from retaliation, and their identities are kept confidential.
As of October 2026, Fangcun AI has not carried out any paid evaluation of a frontier model. In the most recent fiscal year, we received no revenue or donations from frontier AI companies, and no free compute credits or model access beyond public channels, such as early access or internal versions. All models we evaluate are accessed through public APIs at standard prices.
We welcome questions about this policy, and concerns about the independence of any Fangcun AI evaluation, at contact@fangcunleap.com.
We review this policy at least once a year. Material changes are published with a new version number and effective date.
This policy is published in English and Chinese. If the two versions differ, the English version prevails.
Safety evaluations sometimes uncover flaws in AI systems that could cause harm if exploited or left unaddressed. Notifying the developer early, so that a problem can be fixed before it becomes widely known, is a responsible approach. Timely publication also helps the wider industry and the public understand the risks.
This policy sets out how we balance both goals: working with developers to address identified issues while sharing our research openly. It is designed to meet condition 5.4 of AEF-1, the AI Evaluator Forum's minimum operating conditions for independent third-party AI evaluations.
This policy applies to every AI system Fangcun AI evaluates, whether it is publicly available or accessed under an agreement with its developer. It covers findings made by everyone involved in our evaluations, including employees, interns, contractors, and advisors. The Fangcun AI Responsible Disclosure Policy therefore applies more broadly than the Fangcun AI Independence and Conflict of Interest Policy, which covers only evaluations of named frontier AI companies: responsible disclosure is relevant to every system we examine.
We handle findings in two categories.
| Type | What it covers | Who we notify | Timing |
|---|---|---|---|
| Security vulnerability | Reproducible flaws that could enable near-term misuse and have a feasible fix, such as universal jailbreaks, prompt-injection exploits, or ways around safeguards | The developer first. If the flaw may transfer to other systems, we may also notify affected domestic developers or hosting platforms only when necessary to prevent a security incident and with prior approval from our legal team; we do not provide unpublished vulnerability information to any overseas organization or individual other than the developer | Published no later than 60 days after notifying the developer, as a rule; this may be extended by mutual agreement while a fix is underway; where the laws and regulations of the host country impose different requirements, those requirements prevail (see Host-country compliance in Section 5) |
| Behavioral finding | Other undesirable behavior, such as reward hacking, deception, sycophancy, manipulation, or harm to vulnerable users | The developer; advance notice depends on whether it would help address the issue | Determined case by case |
For behavioral findings, we weigh how severe and widespread the potential harm is, whether the developer could fix it before publication, and whether the behavior is already publicly known.
We send findings to the developer's published security or safety contact, with enough detail to reproduce them. Where a developer has no published contact, we use the most direct channel available and keep a record of our attempts.
Developers can reply to a notification at contact@fangcunleap.com. We carefully consider their response, including progress on a fix and any factual corrections, and update our report where appropriate. Decisions on publication are made independently by Fangcun AI.
Security vulnerabilities are published within the timeframe set out in Section 3. Behavioral findings are published when doing so serves public understanding of AI risk. Each published evaluation states whether the developer was notified in advance and, if so, when. For attack paths that are probabilistic, we weigh the marginal utility of publication, including whether publication itself would materially increase the likelihood of misuse, before deciding what to publish.
Developers responding to a notification, and anyone who wants to report a concern about a Fangcun AI evaluation, can write to contact@fangcunleap.com.
We review this policy at least once a year. Material changes are published with a new version number and effective date.
This policy is published in English and Chinese. If the two versions differ, the English version prevails.
These terms govern your use of the services provided by Fangcun AI (方寸跃迁, referred to as “we”). By registering, making a payment or calling an API, you accept these terms.
For questions about these terms, contact info@fangcunleap.com


