{"id":561,"date":"2026-01-31T21:06:13","date_gmt":"2026-01-31T13:06:13","guid":{"rendered":"https:\/\/blog.itsmygo.uk\/2026\/01\/31\/shipping-ai-that-works-an-evaluation-framework-for-pms%ef%bd%9c%e6%89%93%e9%80%a0%e5%8f%af%e9%9d%a0-ai-%e7%94%a2%e5%93%81%ef%bc%9a%e7%94%a2%e5%93%81%e7%b6%93%e7%90%86%e7%9a%84%e8%a9%95%e4%bc%b0\/"},"modified":"2026-01-31T21:06:13","modified_gmt":"2026-01-31T13:06:13","slug":"shipping-ai-that-works-an-evaluation-framework-for-pms%ef%bd%9c%e6%89%93%e9%80%a0%e5%8f%af%e9%9d%a0-ai-%e7%94%a2%e5%93%81%ef%bc%9a%e7%94%a2%e5%93%81%e7%b6%93%e7%90%86%e7%9a%84%e8%a9%95%e4%bc%b0","status":"publish","type":"post","link":"https:\/\/blog.itsmygo.uk\/wordpress\/2026\/01\/31\/shipping-ai-that-works-an-evaluation-framework-for-pms%ef%bd%9c%e6%89%93%e9%80%a0%e5%8f%af%e9%9d%a0-ai-%e7%94%a2%e5%93%81%ef%bc%9a%e7%94%a2%e5%93%81%e7%b6%93%e7%90%86%e7%9a%84%e8%a9%95%e4%bc%b0\/","title":{"rendered":"Shipping AI That Works: An Evaluation Framework for PMs\uff5c\u6253\u9020\u53ef\u9760 AI \u7522\u54c1\uff1a\u7522\u54c1\u7d93\u7406\u7684\u8a55\u4f30\u6846\u67b6"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Arize \u7684 AI \u7522\u54c1\u7d93\u7406 Aman Khan \u5728 AI Engineer World&#8217;s Fair \u7684\u5de5\u4f5c\u574a\u4e2d\uff0c\u5f9e\u81ea\u99d5\u8eca\u5230 LLM Agent \u7684\u7d93\u9a57\u51fa\u767c\uff0c\u7cfb\u7d71\u6027\u5730\u4ecb\u7d39\u4e86 AI \u7522\u54c1\u8a55\u4f30\uff08Eval\uff09\u6846\u67b6\u3002\u4ed6\u5f37\u8abf\u300cVibe Coding\u300d\u4e0d\u8db3\u4ee5\u61c9\u4ed8\u751f\u7522\u74b0\u5883\u7684\u9700\u6c42\uff0c\u63d0\u51fa\u300cThrive Coding\u300d\u7684\u6982\u5ff5\u2014\u2014\u7528\u6578\u64da\u9a45\u52d5\u7684\u8a55\u4f30\u65b9\u5f0f\u53d6\u4ee3\u76f4\u89ba\u5224\u65b7\uff0c\u4e26\u900f\u904e\u73fe\u5834 Demo \u6f14\u793a\u5982\u4f55\u5efa\u7acb\u3001\u57f7\u884c\u548c\u8fed\u4ee3 Eval\uff0c\u5e6b\u52a9 AI \u7522\u54c1\u7d93\u7406\u5728\u5feb\u901f\u8b8a\u5316\u7684 AI \u6642\u4ee3\u4e2d\u4ea4\u4ed8\u53ef\u9760\u7684\u7522\u54c1\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>\u539f\u5f71\u7247\u9023\u7d50\uff1ahttps:\/\/www.youtube.com\/watch?v=2HNSG990Ew8<\/p><\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\">\u5f71\u7247\u91cd\u9ede<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>LLM \u7684\u975e\u78ba\u5b9a\u6027\u672c\u8cea\u4f7f\u5f97\u50b3\u7d71\u8edf\u9ad4\u6e2c\u8a66\u65b9\u6cd5\u4e0d\u9069\u7528\u65bc AI \u7522\u54c1\uff0c\u9700\u8981\u5c08\u9580\u7684\u8a55\u4f30\uff08Eval\uff09\u6846\u67b6<\/li>\n<li>OpenAI \u548c Anthropic \u7684\u7522\u54c1\u8ca0\u8cac\u4eba\u90fd\u516c\u958b\u627f\u8a8d\u6a21\u578b\u6703\u7522\u751f\u5e7b\u89ba\uff0cEval \u6b63\u5728\u6210\u70ba AI \u65b0\u5275\u516c\u53f8\u7684\u771f\u6b63\u8b77\u57ce\u6cb3<\/li>\n<li>\u300cVibe Coding\u300d\u53ea\u9069\u5408\u539f\u578b\u958b\u767c\uff0c\u751f\u7522\u74b0\u5883\u9700\u8981\u300cThrive Coding\u300d\u2014\u2014\u7528\u6578\u64da\u4f86\u9a57\u8b49\u7522\u51fa\u54c1\u8cea<\/li>\n<li>LLM as a Judge \u662f\u76ee\u524d\u6700\u5177\u898f\u6a21\u5316\u6f5b\u529b\u7684\u8a55\u4f30\u65b9\u5f0f\uff0c\u4f46\u4ecd\u9700\u4eba\u985e\u6a19\u8a3b\u4f86\u6821\u9a57 Judge \u7684\u6e96\u78ba\u6027<\/li>\n<li>\u8a55\u4f30\u4e0d\u53ea\u662f\u770b\u55ae\u4e00\u8f38\u51fa\uff0c\u800c\u662f\u8981\u5728\u5b8c\u6574\u8cc7\u6599\u96c6\u4e0a\u9032\u884c A\/B \u6e2c\u8a66\uff0c\u6bd4\u8f03\u4e0d\u540c Prompt \u548c\u6a21\u578b\u7684\u8868\u73fe<\/li>\n<li>\u4f5c\u70ba AI \u7522\u54c1\u7d93\u7406\uff0cEval \u53ef\u4ee5\u53d6\u4ee3\u50b3\u7d71 PRD \u6210\u70ba\u65b0\u578b\u614b\u7684\u9700\u6c42\u898f\u683c\u2014\u2014\u300cEval \u5373\u9700\u6c42\u300d<\/li>\n<li>\u5efa\u7acb Eval \u8cc7\u6599\u96c6\u662f\u6301\u7e8c\u8fed\u4ee3\u7684\u904e\u7a0b\uff0c\u985e\u4f3c\u81ea\u99d5\u8eca\u5f9e\u76f4\u7dda\u884c\u99db\u5230\u8655\u7406\u884c\u4eba\u5de6\u8f49\u7684\u6f38\u9032\u5f0f\u6539\u5584<\/li>\n<li>AI \u6642\u4ee3\u7684\u7522\u54c1\u7d93\u7406\u9700\u8981\u66f4\u9ad8\u7684\u6280\u8853\u7d20\u990a\uff0c\u80fd\u5920\u76f4\u63a5\u4f7f\u7528 Cursor \u7b49\u5de5\u5177\u8207\u7a0b\u5f0f\u78bc\u4e92\u52d5<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">\u8a73\u7d30\u5167\u5bb9<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">[00:00] \u958b\u5834\u8207\u81ea\u6211\u4ecb\u7d39<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Aman Khan \u662f Arize \u7684 AI \u7522\u54c1\u7d93\u7406\uff0c\u64c1\u6709\u5de5\u7a0b\u80cc\u666f\u3002\u4ed6\u7684\u8077\u6daf\u5f9e Cruise \u7684\u81ea\u99d5\u8eca\u8a55\u4f30\u7cfb\u7d71\u958b\u59cb\uff082018-2019\uff09\uff0c\u4e4b\u5f8c\u5230 Spotify \u8ca0\u8cac\u6a5f\u5668\u5b78\u7fd2\u5e73\u53f0\u8207\u63a8\u85a6\u7cfb\u7d71\uff08\u5982 Discover Weekly\u3001\u641c\u5c0b\u7684 Embedding \u61c9\u7528\uff09\uff0c\u6700\u5f8c\u52a0\u5165 Arize \u5df2\u8d85\u904e\u4e09\u5e74\u534a\uff0c\u6301\u7e8c\u5c08\u6ce8\u65bc AI \u8a55\u4f30\u7cfb\u7d71\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Arize \u7684\u5ba2\u6236\u5305\u62ec Uber\u3001Instacart\u3001Reddit\u3001Duolingo \u7b49\u6280\u8853\u524d\u6cbf\u516c\u53f8\uff0c\u5f9e\u50b3\u7d71\u6a5f\u5668\u5b78\u7fd2\uff08\u6392\u5e8f\u3001\u8ff4\u6b78\u3001\u5206\u985e\uff09\u8d77\u6b65\uff0c\u73fe\u5df2\u64f4\u5c55\u5230\u751f\u6210\u5f0f AI \u548c Agent \u61c9\u7528\u9818\u57df\u3002\u4ed6\u5011\u7684\u6838\u5fc3\u4f7f\u547d\u662f\u78ba\u4fdd\u5ba2\u6236\u7684 AI \u61c9\u7528\u548c Agent \u80fd\u5982\u9810\u671f\u904b\u4f5c\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[02:30] \u70ba\u4ec0\u9ebc Eval \u5982\u6b64\u91cd\u8981<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Aman \u5f15\u7528\u4e86\u591a\u4f4d\u696d\u754c\u9818\u8896\u7684\u89c0\u9ede\u4f86\u5f37\u8abf Eval \u7684\u91cd\u8981\u6027\uff1aOpenAI \u7684\u9996\u5e2d\u7522\u54c1\u9577 Kevin Weil\u3001Anthropic \u7684 CPO Mike Krieger \u90fd\u5728 Lenny&#8217;s Conference \u4e0a\u516c\u958b\u8868\u793a\u4ed6\u5011\u7684\u6a21\u578b\u6703\u7522\u751f\u5e7b\u89ba\uff0c\u4e26\u5f37\u8abf\u64b0\u5beb Eval \u7684\u91cd\u8981\u6027\u3002Greg Brockman\uff08OpenAI \u5171\u540c\u5275\u8fa6\u4eba\uff09\u548c\u5176\u4ed6\u696d\u754c\u4eba\u58eb\u4e5f\u6307\u51fa\u300cEval \u6b63\u5728\u6210\u70ba AI \u65b0\u5275\u516c\u53f8\u7684\u771f\u6b63\u8b77\u57ce\u6cb3\u300d\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u95dc\u9375\u8a0a\u606f\u662f\uff1a\u7576\u8ce3\u7d66\u4f60\u7522\u54c1\u7684\u4eba\u90fd\u544a\u8a34\u4f60\u7522\u54c1\u4e0d\u5b8c\u5168\u53ef\u9760\u6642\uff0c\u4f60\u78ba\u5be6\u61c9\u8a72\u8a8d\u771f\u770b\u5f85\u3002\u8a31\u591a\u4f86\u81ea\u81ea\u99d5\u8eca\u9818\u57df\u7684\u7d93\u9a57\u6559\u8a13\u540c\u6a23\u9069\u7528\u65bc\u7576\u4eca\u7684 AI \u61c9\u7528\u958b\u767c\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[05:00] AI \u7522\u54c1\u7d93\u7406\u7684\u5b78\u7fd2\u66f2\u7dda<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Aman \u7528\u985e\u4f3c Dunning-Kruger \u6548\u61c9\u7684\u66f2\u7dda\u4f86\u63cf\u8ff0 AI PM \u7684\u6210\u9577\u65c5\u7a0b\u3002\u5373\u4fbf\u4ed6\u672c\u4eba\u5728\u4e00\u5bb6 Eval \u516c\u53f8\u5de5\u4f5c\uff0c\u4e5f\u7d93\u6b77\u4e86\u540c\u6a23\u7684\u904e\u7a0b\uff1a\u5f9e\u5b78\u7fd2\u5982\u4f55\u5728\u5de5\u4f5c\u4e2d\u4f7f\u7528 AI\uff0c\u5230\u7528 AI \u5de5\u5177\u5efa\u7acb\u539f\u578b\uff0c\u518d\u5230\u9762\u5c0d\u300c\u5982\u4f55\u5c07\u5e36\u6709 LLM \u6216 Agent \u7684\u7522\u54c1\u63a8\u5411\u751f\u7522\u74b0\u5883\u300d\u7684\u6311\u6230\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5728\u9019\u500b\u904e\u7a0b\u4e2d\uff0c\u4fe1\u5fc3\u6703\u7d93\u6b77\u4e00\u500b\u4f4e\u8c37\u2014\u2014\u4f60\u6703\u767c\u73fe\u7f3a\u4e4f\u5de5\u5177\u652f\u63f4\u3001\u7f3a\u4e4f\u95dc\u65bc\u5982\u4f55\u53ef\u9760\u5efa\u69cb\u9019\u4e9b\u7cfb\u7d71\u7684\u6559\u80b2\u8cc7\u6e90\u3002\u7522\u54c1\u7ba1\u7406\u89d2\u8272\u7684\u671f\u671b\u6b63\u5728\u6539\u8b8a\uff0c\u300cAIPM\u300d\uff08AI \u7522\u54c1\u7d93\u7406\uff09\u7684\u6982\u5ff5\u6b63\u5728\u8208\u8d77\uff0c\u4f86\u81ea\u5229\u5bb3\u95dc\u4fc2\u4eba\u548c\u5de5\u7a0b\u5e2b\u7684\u671f\u671b\u4e5f\u5728\u4e0d\u65b7\u63d0\u9ad8\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[07:30] \u4ec0\u9ebc\u662f Eval\u2014\u2014\u8207\u50b3\u7d71\u8edf\u9ad4\u6e2c\u8a66\u7684\u95dc\u9375\u5dee\u7570<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Eval \u985e\u4f3c\u65bc\u8edf\u9ad4\u6e2c\u8a66\uff0c\u4f46\u6709\u5e7e\u500b\u95dc\u9375\u5dee\u7570\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>\u78ba\u5b9a\u6027 vs \u975e\u78ba\u5b9a\u6027<\/strong>\uff1a\u8edf\u9ad4\u4e2d 1+1 \u6c38\u9060\u7b49\u65bc 2\uff0c\u4f46\u5982\u679c\u4f60\u8aaa\u670d LLM Agent 1+1=3\uff0c\u5b83\u6703\u8d0a\u540c\u4f60\u3002\u9019\u4e9b\u7cfb\u7d71\u9ad8\u5ea6\u53ef\u64cd\u63a7\u3002<\/li>\n<li><strong>\u591a\u8def\u5f91\u57f7\u884c<\/strong>\uff1aLLM Agent \u53ef\u4ee5\u8d70\u4e0d\u540c\u8def\u5f91\uff0c\u9019\u548c\u78ba\u5b9a\u6027\u7684\u55ae\u5143\u6e2c\u8a66\u5b8c\u5168\u4e0d\u540c\u3002\u4f60\u5176\u5be6\u5e0c\u671b Agent\u300c\u4ee5\u6b63\u78ba\u7684\u65b9\u5f0f\u7522\u751f\u5e7b\u89ba\u300d\uff0c\u9019\u8b93\u6e2c\u8a66\u66f4\u52a0\u56f0\u96e3\u3002<\/li>\n<li><strong>\u4f9d\u8cf4\u6578\u64da\u800c\u975e\u7a0b\u5f0f\u78bc<\/strong>\uff1a\u6574\u5408\u6e2c\u8a66\u4f9d\u8cf4\u65e2\u6709\u7684\u7a0b\u5f0f\u78bc\u548c\u6587\u4ef6\uff0c\u4f46 Agent \u4f9d\u8cf4\u7684\u662f\u4f60\u7684\u6578\u64da\u3002\u4f7f\u7528\u8005\u9078\u64c7\u4f60\u7684 Agent \u800c\u975e\u5176\u4ed6\u65b9\u6848\uff0c\u5f88\u5927\u7a0b\u5ea6\u4e0a\u662f\u56e0\u70ba\u4f60\u7684\u6578\u64da\u3002<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">[09:30] Eval \u7684\u56db\u500b\u7d44\u6210\u90e8\u5206<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u4e00\u500b Eval \u7531\u56db\u500b\u90e8\u5206\u7d44\u6210\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>\u89d2\u8272\uff08Role\uff09<\/strong>\uff1a\u544a\u8a34 Agent \u8981\u5b8c\u6210\u4ec0\u9ebc\u4efb\u52d9<\/li>\n<li><strong>\u4e0a\u4e0b\u6587\uff08Context\uff09<\/strong>\uff1a\u63d0\u4f9b\u8981\u8a55\u4f30\u7684\u6587\u672c\u5167\u5bb9<\/li>\n<li><strong>\u76ee\u6a19\uff08Goal\uff09<\/strong>\uff1aAgent \u8981\u5224\u65b7\u7684\u6a19\u6e96\uff08\u4f8b\u5982\u6587\u672c\u662f\u5426\u6709\u6bd2\uff09<\/li>\n<li><strong>\u8853\u8a9e\u8207\u6a19\u7c64\uff08Terminology &#038; Label\uff09<\/strong>\uff1a\u7d66\u51fa\u597d\u58de\u7684\u7bc4\u4f8b\u548c\u8f38\u51fa\u9078\u9805<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">\u91cd\u8981\u89c0\u5ff5\uff1a\u4e0d\u8981\u8b93 LLM \u76f4\u63a5\u7522\u751f\u6578\u5b57\u5206\u6578\uff0c\u56e0\u70ba\u5373\u4f7f\u662f PhD \u7b49\u7d1a\u7684 LLM \u5728\u8655\u7406\u6578\u5b57\u6642\u4ecd\u7136\u5f88\u4e0d\u64c5\u9577\u3002\u61c9\u8a72\u4f7f\u7528\u6587\u5b57\u6a19\u7c64\uff08\u5982\u300ctoxic \/ not toxic\u300d\uff09\uff0c\u518d\u5c07\u6a19\u7c64\u6620\u5c04\u5230\u5206\u6578\u3002Arize \u6709\u7814\u7a76\u8b49\u5be6\u9019\u5728\u5927\u898f\u6a21\u548c\u591a\u6578\u6a21\u578b\u4e0a\u90fd\u6210\u7acb\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[12:30] \u5f9e Vibe Coding \u5230 Thrive Coding<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u300cVibe Coding\u300d\u7684\u73fe\u72c0\u662f\uff1a\u5927\u5bb6\u7528 Bolt\u3001Lovable \u7b49\u5de5\u5177\u5beb\u7a0b\u5f0f\u78bc\uff0c\u770b\u8d77\u4f86\u5dee\u4e0d\u591a\u5c31\u884c\u4e86\uff0c\u4e0d\u592a\u6703\u4ed4\u7d30\u5be9\u67e5 AI \u751f\u6210\u7684\u7a0b\u5f0f\u78bc\u3002\u9019\u5728\u539f\u578b\u958b\u767c\u968e\u6bb5\u6c92\u554f\u984c\uff0c\u4f46\u5728\u751f\u7522\u74b0\u5883\u4e2d\u884c\u4e0d\u901a\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Aman \u63d0\u51fa\u7528\u300cThrive Coding\u300d\u53d6\u4ee3\u2014\u2014\u6838\u5fc3\u6982\u5ff5\u662f\u4ecd\u7136\u5feb\u901f\u5efa\u69cb\u61c9\u7528\uff0c\u4f46\u7528\u6578\u64da\u4f86\u9a57\u8b49\u8f38\u51fa\u54c1\u8cea\uff0c\u5f9e\u800c\u5c0d\u7522\u51fa\u6709\u66f4\u9ad8\u7684\u4fe1\u5fc3\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[14:00] \u73fe\u5834 Demo\uff1aAI \u65c5\u884c\u898f\u5283\u5668<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Aman \u5c55\u793a\u4e86\u4e00\u500b\u7528 LangGraph \u548c\u591a Agent \u7cfb\u7d71\u5efa\u69cb\u7684 AI \u65c5\u884c\u898f\u5283\u5668\u3002\u9019\u500b\u61c9\u7528\u63a5\u6536\u76ee\u7684\u5730\u3001\u5929\u6578\u3001\u9810\u7b97\u548c\u8208\u8da3\u7b49\u8f38\u5165\uff0c\u7531\u591a\u500b Agent \u5354\u4f5c\u7522\u751f\u5b8c\u6574\u7684\u65c5\u884c\u884c\u7a0b\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u7cfb\u7d71\u5e95\u4e0b\u6709\u56db\u500b Agent \u5354\u540c\u5de5\u4f5c\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>\u9810\u7b97 Agent<\/strong>\uff1a\u8655\u7406\u8cbb\u7528\u8a08\u7b97<\/li>\n<li><strong>\u5728\u5730\u9ad4\u9a57 Agent<\/strong>\uff1a\u5c0b\u627e\u7576\u5730\u7279\u8272\u6d3b\u52d5<\/li>\n<li><strong>\u7814\u7a76 Agent<\/strong>\uff1a\u6536\u96c6\u76ee\u7684\u5730\u8cc7\u8a0a<\/li>\n<li><strong>\u884c\u7a0b Agent<\/strong>\uff1a\u6574\u5408\u4ee5\u4e0a\u4e09\u500b Agent \u7684\u8f38\u51fa\uff0c\u7522\u751f\u6700\u7d42\u884c\u7a0b<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">[18:00] \u53ef\u89c0\u6e2c\u6027\uff08Observability\uff09\uff1a\u770b\u898b Agent \u7684\u904b\u4f5c<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u900f\u904e Arize \u5e73\u53f0\uff08\u4e5f\u6709\u958b\u6e90\u7248 Phoenix\uff09\uff0c\u53ef\u4ee5\u8996\u89ba\u5316 Agent \u7cfb\u7d71\u7684\u904b\u4f5c\u65b9\u5f0f\u3002\u6bcf\u500b\u8acb\u6c42\u6703\u7522\u751f Trace\uff08\u8ffd\u8e64\u8a18\u9304\uff09\uff0c\u7531\u591a\u500b Span\uff08\u5de5\u4f5c\u55ae\u5143\uff09\u7d44\u6210\u3002Span \u6709\u4e09\u7a2e\u985e\u578b\uff1aAgent\u3001Tool\uff08\u4f7f\u7528\u7d50\u69cb\u5316\u6578\u64da\u57f7\u884c\u52d5\u4f5c\uff09\u548c LLM\uff08\u6839\u64da\u8f38\u5165\u548c\u4e0a\u4e0b\u6587\u7522\u751f\u8f38\u51fa\uff09\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5c0d\u7522\u54c1\u7d93\u7406\u800c\u8a00\uff0c\u80fd\u5920\u56de\u5230\u5718\u968a\u4e26\u8a62\u554f\u300c\u6211\u5011\u7684 Agent \u5be6\u969b\u4e0a\u9577\u4ec0\u9ebc\u6a23\u300d\uff0c\u770b\u5230\u7cfb\u7d71\u7684\u8996\u89ba\u5316\u8868\u793a\uff0c\u9019\u662f\u975e\u5e38\u6709\u50f9\u503c\u7684\u3002Arize \u65b0\u63a8\u51fa\u7684 Agent \u8996\u89ba\u5316\u529f\u80fd\u53ef\u4ee5\u6e05\u695a\u5448\u73fe\u5e73\u884c\u547c\u53eb\u548c Agent \u9593\u7684\u8cc7\u6599\u6d41\u52d5\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[22:00] Prompt Playground \u8207\u8fed\u4ee3<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Arize \u63d0\u4f9b Prompt Playground\uff0c\u53ef\u4ee5\u5c07\u8ffd\u8e64\u5230\u7684 Prompt \u53ca\u5176\u8b8a\u6578\u62c9\u5165\u4e92\u52d5\u5f0f\u74b0\u5883\u9032\u884c\u4fee\u6539\u548c\u6e2c\u8a66\u3002\u9019\u8b93\u7522\u54c1\u7d93\u7406\u80fd\u5920\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u76f4\u63a5\u4fee\u6539\u76ee\u7684\u5730\u3001\u884c\u7a0b\u5929\u6578\u7b49\u8b8a\u6578<\/li>\n<li>\u5207\u63db\u4e0d\u540c LLM \u6a21\u578b\uff08\u5982\u5f9e GPT-4o \u63db\u6210 4.1 Mini\uff09<\/li>\n<li>\u6dfb\u52a0\u65b0\u7684 Prompt \u6307\u4ee4\uff08\u5982\u300c\u4fdd\u6301\u5728 500 \u5b57\u4ee5\u5167\u300d\u3001\u300c\u7528\u8d85\u7d1a\u53cb\u5584\u7684\u8a9e\u6c23\u300d\u3001\u300c\u63d0\u4f9b\u6298\u6263\u4ee5\u63db\u53d6\u7528\u6236 Email\u300d\uff09<\/li>\n<li>\u7248\u672c\u63a7\u5236 Prompt\uff0c\u985e\u4f3c GitHub \u5132\u5b58\u5eab<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">\u4e00\u500b\u503c\u5f97\u601d\u8003\u7684\u554f\u984c\uff1a\u64b0\u5beb Prompt \u61c9\u8a72\u662f\u5de5\u7a0b\u5e2b\u9084\u662f\u7522\u54c1\u7d93\u7406\u7684\u8cac\u4efb\uff1f\u5982\u679c\u4f60\u662f\u7522\u54c1\u8ca0\u8cac\u4eba\uff0c\u5c0d\u6700\u7d42\u7522\u54c1\u7d50\u679c\u8ca0\u8cac\uff0c\u4f60\u53ef\u80fd\u9700\u8981\u5c0d Prompt \u6709\u66f4\u591a\u7684\u638c\u63a7\u6b0a\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[28:00] \u5f9e\u55ae\u4e00\u7bc4\u4f8b\u5230\u8cc7\u6599\u96c6\u2014\u2014\u898f\u6a21\u5316\u8a55\u4f30<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u7528\u55ae\u4e00\u7bc4\u4f8b\u505a\u300c\u597d\u6216\u58de\u300d\u7684\u5224\u65b7\u7121\u6cd5\u898f\u6a21\u5316\u3002\u5718\u968a\u9700\u8981\u5efa\u7acb\u5305\u542b 10 \u500b\u3001100 \u500b\u751a\u81f3\u66f4\u591a\u7bc4\u4f8b\u7684\u8cc7\u6599\u96c6\uff0c\u7136\u5f8c\u5728\u6574\u500b\u8cc7\u6599\u96c6\u4e0a\u9032\u884c A\/B \u6e2c\u8a66\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u73fe\u5834\u6f14\u793a\u4e2d\uff0cAman \u5efa\u7acb\u4e86\u5169\u500b\u5be6\u9a57\uff1aPrompt A\uff08\u539f\u59cb\u7248\u672c\uff09\u548c Prompt B\uff08\u4fee\u6539\u7248\u2014\u2014\u9650\u5236\u5b57\u6578\u3001\u8981\u6c42\u53cb\u5584\u8a9e\u6c23\u3001\u8981\u6c42\u63d0\u4f9b\u6298\u6263\uff09\u3002\u7d50\u679c\u986f\u793a\u4fee\u6539\u5f8c\u7684 Prompt \u56de\u61c9\u901f\u5ea6\u66f4\u5feb\uff08\u56e0\u70ba\u9650\u5236\u4e86\u5b57\u6578\uff09\uff0c\u4f46\u539f\u59cb Prompt \u5e73\u5747\u8981 32 \u79d2\u624d\u80fd\u5b8c\u6210\uff0c\u56e0\u70ba\u6c92\u6709\u9650\u5236\u8f38\u51fa\u9577\u5ea6\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[33:00] \u5efa\u7acb\u8207\u57f7\u884c Eval<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Aman \u5c55\u793a\u4e86\u5169\u500b Eval \u7684\u5be6\u969b\u5efa\u7acb\u904e\u7a0b\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>\u53cb\u5584\u5ea6 Eval<\/strong>\uff1a\u5224\u65b7 Agent \u56de\u61c9\u662f\u300cfriendly\u300d\u9084\u662f\u300crobotic\u300d\uff0c\u5b9a\u7fa9\u4e86\u53cb\u5584\u8a9e\u6c23\u7684\u5177\u9ad4\u6a19\u6e96\uff08upbeat\u3001cheerful \u7b49\uff09<\/li>\n<li><strong>\u6298\u6263 Eval<\/strong>\uff1a\u6aa2\u67e5\u56de\u61c9\u4e2d\u662f\u5426\u5305\u542b\u6298\u6263\u512a\u60e0<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">\u57f7\u884c\u7d50\u679c\u986f\u793a\uff1a\u4fee\u6539\u5f8c\u7684 Prompt B \u5728\u6240\u6709\u7bc4\u4f8b\u4e2d\u90fd\u6210\u529f\u63d0\u4f9b\u4e86\u6298\u6263\uff08100% \u901a\u904e\uff09\u3002\u539f\u59cb Prompt A \u6c92\u6709\u6298\u6263\uff0c\u4f46 LLM Judge \u4ecd\u5c07\u5176\u8a55\u70ba\u300cfriendly\u300d\u2014\u2014\u800c Aman \u500b\u4eba\u4e26\u4e0d\u540c\u610f\u9019\u500b\u5224\u65b7\uff0c\u56e0\u70ba\u539f\u59cb\u56de\u61c9\u592a\u5197\u9577\u3001\u592a\u50cf\u6a5f\u5668\u751f\u6210\u7684\u6587\u5b57\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[38:00] \u70ba Eval \u52a0\u4e0a\u4eba\u985e\u6821\u9a57\u2014\u2014Eval \u4f60\u7684 Eval<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u6838\u5fc3\u89c0\u5ff5\uff1a\u4f60\u4e0d\u80fd\u53ea\u4fe1\u4efb LLM as a Judge \u7684\u7d50\u679c\uff0c\u9084\u9700\u8981\u4eba\u985e\u6a19\u8a3b\u4f86\u9a57\u8b49\u3002\u6d41\u7a0b\u662f\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u5c07\u8cc7\u6599\u96c6\u5206\u914d\u7d66\u4e3b\u984c\u5c08\u5bb6\u9032\u884c\u4eba\u5de5\u6a19\u8a3b<\/li>\n<li>\u7528\u7a0b\u5f0f\u5316 Eval\uff08\u5982\u5b57\u4e32\u5339\u914d\uff09\u6bd4\u8f03\u4eba\u985e\u6a19\u7c64\u548c LLM \u6a19\u7c64<\/li>\n<li>\u627e\u51fa\u5169\u8005\u4e0d\u4e00\u81f4\u7684\u5730\u65b9\uff0c\u6539\u9032 Eval Prompt<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">\u5728\u6f14\u793a\u4e2d\uff0c\u6bd4\u8f03\u7d50\u679c\u986f\u793a LLM Judge \u7684\u53cb\u5584\u5ea6\u5224\u65b7\u5e7e\u4e4e\u5b8c\u5168\u4e0d\u7b26\u5408\u4eba\u985e\u6a19\u7c64\u2014\u2014\u9019\u6b63\u662f\u9700\u8981\u6539\u9032 Eval \u7684\u4fe1\u865f\u3002\u6539\u9032\u65b9\u5f0f\u5305\u62ec\uff1a\u589e\u52a0 Few-Shot \u7bc4\u4f8b\u3001\u66f4\u56b4\u683c\u5b9a\u7fa9\u300cfriendly\u300d\u7684\u6a19\u6e96\u3001\u964d\u4f4e Temperature \u4ee5\u6e1b\u5c11\u8b8a\u7570\u3001\u591a\u6b21\u57f7\u884c\u53d6\u5e73\u5747\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Arize \u5e73\u53f0\u4e5f\u63d0\u4f9b Copilot \u529f\u80fd\uff0c\u53ef\u4ee5\u5e6b\u52a9\u64b0\u5beb\u548c\u512a\u5316 Eval Prompt\u3002\u91cd\u8981\u7684\u662f\uff1a\u4f60\u53ef\u80fd\u9700\u8981 5-10 \u6b21\u8fed\u4ee3\u624d\u80fd\u5f97\u5230\u4e00\u500b\u8207\u4eba\u985e\u6a19\u7c64\u543b\u5408\u7684 Eval\uff0c\u9019\u662f\u6b63\u5e38\u7684\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[44:00] Eval \u5373\u9700\u6c42\u2014\u2014AI PM \u7684\u65b0\u5b9a\u4f4d<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Aman \u8a8d\u70ba Eval \u6b63\u5728\u6210\u70ba\u4e00\u7a2e\u65b0\u578b\u614b\u7684\u9700\u6c42\u898f\u683c\u3002\u60f3\u50cf\u5982\u679c\u4f60\u80fd\u7528 Eval \u8cc7\u6599\u96c6\u548c\u9a57\u6536\u6a19\u6e96\u53d6\u4ee3 PRD \u4ea4\u7d66\u5de5\u7a0b\u5718\u968a\uff0c\u9019\u5c07\u5927\u5927\u63d0\u5347\u6e9d\u901a\u6548\u7387\u548c\u7522\u54c1\u54c1\u8cea\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u958b\u767c\u9031\u671f\u7684\u5faa\u74b0\u662f\uff1a\u958b\u767c\u968e\u6bb5\u7528 CSV \u8cc7\u6599\u548c\u624b\u52d5\u7bc4\u4f8b \u2192 \u6301\u7e8c\u7b56\u5c55\u8cc7\u6599\u96c6\u548c\u91cd\u8dd1\u5be6\u9a57 \u2192 \u5718\u968a\u6709\u4fe1\u5fc3\u5f8c\u63a8\u5411\u751f\u7522 \u2192 \u5728\u751f\u7522\u74b0\u5883\u4e2d\u7e7c\u7e8c\u7528\u771f\u5be6\u6578\u64da\u8a55\u4f30 \u2192 \u5c07\u65b0\u767c\u73fe\u7684\u554f\u984c\u7bc4\u4f8b\u56de\u994b\u5230\u958b\u767c\u74b0\u7bc0\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[47:00] \u81ea\u99d5\u8eca\u7684\u555f\u793a\u2014\u2014\u6f38\u9032\u5f0f\u6539\u5584<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Aman \u7528\u81ea\u99d5\u8eca\u7684\u7d93\u9a57\u505a\u6bd4\u55bb\uff1a\u5728 Cruise \u6642\uff0c\u8eca\u5b50\u4e00\u958b\u59cb\u9023\u4e00\u500b\u8857\u5340\u90fd\u958b\u4e0d\u5b8c\u3002\u5148\u89e3\u6c7a\u76f4\u7dda\u884c\u99db\uff0c\u7136\u5f8c\u5de6\u8f49\u6210\u4e86\u554f\u984c\uff0c\u5c31\u5efa\u7acb\u300c\u5de6\u8f49\u300d\u8cc7\u6599\u96c6\u3002\u5de6\u8f49\u904e\u95dc\u5f8c\uff0c\u9047\u5230\u884c\u4eba\u5728\u4eba\u884c\u9053\u4e0a\u7684\u60c5\u5883\uff0c\u53c8\u9700\u8981\u5efa\u7acb\u300c\u5de6\u8f49 + \u884c\u4eba\u300d\u7684\u8cc7\u6599\u96c6\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u95dc\u9375\u555f\u793a\uff1a\u4f60\u4e0d\u6703\u4e8b\u5148\u77e5\u9053\u6240\u6709\u56f0\u96e3\u5834\u666f\uff0c\u53ea\u6709\u5728\u5be6\u969b\u9047\u5230\u6642\u624d\u6703\u767c\u73fe\u3002\u6240\u4ee5\u8981\u63a5\u53d7 Eval \u8cc7\u6599\u96c6\u6703\u96a8\u6642\u9593\u4e0d\u65b7\u6f14\u9032\u7684\u4e8b\u5be6\uff0c\u6301\u7e8c\u52a0\u5165\u65b0\u7684\u300c\u56f0\u96e3\u7bc4\u4f8b\u300d\uff08hard examples\uff09\u2014\u2014\u90a3\u4e9b\u6a21\u578b\u5224\u65b7\u6a21\u7cca\u3001\u4eba\u985e\u4e5f\u89ba\u5f97\u4e3b\u89c0\u7684\u908a\u754c\u6848\u4f8b\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">[53:00] Q&#038;A \u7cbe\u83ef\uff1a\u5718\u968a\u5efa\u8a2d\u8207 PM \u89d2\u8272\u6f14\u8b8a<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u5de5\u4f5c\u574a\u7684 Q&#038;A \u74b0\u7bc0\u6db5\u84cb\u4e86\u591a\u500b\u5be6\u52d9\u8b70\u984c\uff1a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u5982\u4f55\u5efa\u7acb Eval \u5718\u968a<\/strong>\uff1a\u5efa\u8b70\u5148\u89aa\u81ea\u9ad4\u9a57\u64b0\u5beb Eval \u7684\u75db\u9ede\uff0c\u4e86\u89e3\u4ec0\u9ebc\u662f\u56f0\u96e3\u7684\uff0c\u518d\u53bb\u9762\u8a66\u548c\u62db\u8058\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>PM \u5982\u4f55\u8b8a\u5f97\u66f4\u6280\u8853\u5316<\/strong>\uff1a\u5728\u516c\u53f8\u5167\u90e8\u63a8\u52d5 AI \u5de5\u5177\u4f7f\u7528\uff08\u5982\u8b93 PM \u7528 Cursor \u5b58\u53d6\u7a0b\u5f0f\u78bc\uff09\uff0c\u900f\u904e\u5efa\u69cb\u539f\u578b\u5c55\u793a\u53ef\u80fd\u6027\uff0c\u9032\u800c\u5f71\u97ff\u5718\u968a\u6587\u5316\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u6280\u8853\u4eba\u54e1\u662f\u5426\u4e5f\u5728\u8b8a\u5f97\u66f4\u50cf PM<\/strong>\uff1a\u7576\u5efa\u69cb\u6210\u672c\u4e0b\u964d\u6642\uff0c\u300c\u5efa\u69cb\u4ec0\u9ebc\u300d\u8b8a\u5f97\u6bd4\u300c\u5982\u4f55\u5efa\u69cb\u300d\u66f4\u91cd\u8981\u3002\u672a\u4f86\u53ef\u80fd\u4e0d\u518d\u4ee5\u89d2\u8272\u5b9a\u7fa9\u81ea\u5df1\uff0c\u800c\u662f\u4ee5\u6280\u80fd\u7d44\u5408\uff08Skill Stack\uff09\u4f86\u4e92\u88dc\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Human in the Loop \u7684\u65b0\u6a21\u5f0f<\/strong>\uff1a\u6709\u516c\u53f8\u6b63\u5728\u5617\u8a66\u5c07\u4eba\u985e\u8a2d\u5b9a\u70ba Agent \u7684\u5de5\u5177\u2014\u2014\u7576 Agent \u9047\u5230\u7121\u6cd5\u8655\u7406\u7684\u4e8b\u52d9\u6642\uff0c\u81ea\u52d5\u900f\u904e Slack \u767c\u9001\u8a0a\u606f\u7d66 CFO \u7b49\u4eba\u7372\u53d6\u8cc7\u8a0a\uff0c\u518d\u7e7c\u7e8c\u57f7\u884c\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u6211\u7684\u60f3\u6cd5<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">\u9019\u5834\u5de5\u4f5c\u574a\u6700\u6253\u52d5\u6211\u7684\u89c0\u9ede\u662f\u300cEval \u5373\u9700\u6c42\u300d\u7684\u6982\u5ff5\u3002\u50b3\u7d71\u4e0a\u7522\u54c1\u7d93\u7406\u7528 PRD\uff08\u7522\u54c1\u9700\u6c42\u6587\u4ef6\uff09\u548c User Story \u4f86\u898f\u7bc4\u7522\u54c1\u884c\u70ba\uff0c\u4f46\u5728 AI \u7522\u54c1\u4e2d\uff0c\u81ea\u7136\u8a9e\u8a00\u63cf\u8ff0\u7684\u9700\u6c42\u5929\u7136\u5730\u5145\u6eff\u6a21\u7cca\u6027\u2014\u2014\u800c Eval \u63d0\u4f9b\u4e86\u4e00\u7a2e\u66f4\u7cbe\u78ba\u3001\u53ef\u91cf\u5316\u7684\u66ff\u4ee3\u65b9\u6848\u3002\u5982\u679c\u5718\u968a\u80fd\u5920\u5171\u540c\u5b9a\u7fa9\u300c\u4ec0\u9ebc\u7b97\u597d\u7684\u8f38\u51fa\u300d\u4e26\u7528\u8cc7\u6599\u96c6\u9a57\u8b49\uff0c\u9019\u6bd4\u4efb\u4f55\u6587\u4ef6\u90fd\u66f4\u6709\u8aaa\u670d\u529b\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u53e6\u4e00\u500b\u503c\u5f97\u95dc\u6ce8\u7684\u6d1e\u5bdf\u662f LLM as a Judge \u7684\u53ef\u4fe1\u5ea6\u554f\u984c\u3002\u6f14\u793a\u4e2d LLM Judge \u5e7e\u4e4e\u5b8c\u5168\u4e0d\u7b26\u5408\u4eba\u985e\u7684\u53cb\u5584\u5ea6\u5224\u65b7\uff0c\u9019\u63d0\u9192\u6211\u5011\u5373\u4f7f\u5728\u300c\u7528 AI \u76e3\u7763 AI\u300d\u7684\u6a21\u5f0f\u4e0b\uff0c\u4eba\u985e\u6821\u9a57\u4ecd\u7136\u4e0d\u53ef\u6216\u7f3a\u3002\u5c0d\u65bc\u9ad8\u98a8\u96aa\u9818\u57df\uff08\u91ab\u7642\u3001\u6cd5\u5f8b\u3001\u91d1\u878d\uff09\uff0c\u9019\u500b\u4eba\u6a5f\u9a57\u8b49\u7684\u9589\u74b0\u66f4\u662f\u4e0d\u80fd\u7701\u7565\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u6700\u5f8c\uff0c\u81ea\u99d5\u8eca\u6f38\u9032\u5f0f\u6539\u5584\u7684\u6bd4\u55bb\u975e\u5e38\u5be6\u7528\u3002\u8a31\u591a\u5718\u968a\u5728 AI \u5c08\u6848\u4e2d\u8ffd\u6c42\u300c\u4e00\u6b21\u5230\u4f4d\u300d\u7684\u5b8c\u7f8e Eval\uff0c\u7d50\u679c\u53cd\u800c\u62d6\u5ef6\u4e86\u4e0a\u7dda\u3002\u66f4\u52d9\u5be6\u7684\u505a\u6cd5\u662f\uff1a\u5148\u7528\u57fa\u672c\u7684 Eval \u9054\u5230\u300c\u8db3\u5920\u4fe1\u5fc3\u300d\u5c31\u4e0a\u7dda\uff0c\u7136\u5f8c\u5728\u751f\u7522\u74b0\u5883\u4e2d\u6301\u7e8c\u6536\u96c6\u56f0\u96e3\u6848\u4f8b\uff0c\u9010\u6b65\u63d0\u9ad8\u7cfb\u7d71\u53ef\u9760\u6027\u3002\u9019\u7a2e\u8fed\u4ee3\u5fc3\u614b\u53ef\u80fd\u662f AI PM \u6700\u9700\u8981\u57f9\u990a\u7684\u6838\u5fc3\u80fd\u529b\u3002<\/p>\n\n    \n\n<div id=\"quiz-3865dcd9\">\n<style>\n#quiz-3865dcd9 .badge {\n        display: inline-block;\n        padding: 4px 10px;\n        border-radius: 12px;\n        font-size: 0.75em;\n        font-weight: bold;\n        margin-left: 8px;\n      }#quiz-3865dcd9 .badge.scenario {\n        background: #ebf8ff;\n        color: #2b6cb0;\n      }#quiz-3865dcd9 .badge.debug {\n        background: #fef3c7;\n        color: #92400e;\n      }#quiz-3865dcd9 .question-adv {\n        background: white;\n        border-radius: 10px;\n        padding: 24px;\n        margin: 20px 0;\n        box-shadow: 0 2px 8px rgba(0, 0, 0, 0.08);\n        border-left: 4px solid #ed8936;\n      }#quiz-3865dcd9 .question-adv h3 {\n        margin-top: 0;\n        color: #2d3748;\n      }#quiz-3865dcd9 .code-block {\n        background: #1e1e1e;\n        color: #d4d4d4;\n        padding: 14px;\n        border-radius: 6px;\n        font-family: \"Menlo\", \"Monaco\", \"Courier New\", monospace;\n        font-size: 0.9em;\n        overflow-x: auto;\n        margin: 12px 0;\n        white-space: pre-wrap;\n      }#quiz-3865dcd9 .options-adv {\n        list-style: none;\n        padding: 0;\n      }#quiz-3865dcd9 .options-adv li {\n        padding: 14px 16px;\n        margin: 10px 0;\n        border: 2px solid #e2e8f0;\n        border-radius: 8px;\n        cursor: pointer;\n        transition: all 0.2s;\n      }#quiz-3865dcd9 .options-adv li:hover:not(.disabled) {\n        border-color: #ed8936;\n        background: #fffaf0;\n      }#quiz-3865dcd9 .options-adv li.disabled {\n        cursor: default;\n        opacity: 0.7;\n      }#quiz-3865dcd9 .options-adv li.show-answer {\n        border-color: #48bb78;\n        background: #f0fff4;\n      }#quiz-3865dcd9 .options-adv li.wrong {\n        border-color: #fc8181;\n        background: #fff5f5;\n      }#quiz-3865dcd9 .feedback-adv {\n        margin-top: 16px;\n        padding: 14px;\n        border-radius: 8px;\n        display: none;\n        line-height: 1.6;\n      }#quiz-3865dcd9 .feedback-adv.correct {\n        background: #f0fff4;\n        color: #276749;\n        display: block;\n      }#quiz-3865dcd9 .feedback-adv.wrong {\n        background: #fff5f5;\n        color: #c53030;\n        display: block;\n      }#quiz-3865dcd9 .feedback-adv \n      .score-adv {\n        text-align: center;\n        font-size: 1.4em;\n        padding: 24px;\n        background: white;\n        border-radius: 10px;\n        margin-top: 24px;\n        box-shadow: 0 2px 8px rgba(0, 0, 0, 0.08);\n      }#quiz-3865dcd9 .intro {\n        background: #ebf8ff;\n        padding: 16px;\n        border-radius: 8px;\n        margin-bottom: 20px;\n        color: #2c5282;\n      }\n<\/style>\n<h1>\u9032\u968e\u6e2c\u9a57\uff1aAI \u7522\u54c1\u7d93\u7406\u7684\u8a55\u4f30\u6846\u67b6<\/h1>\n    <div class=\"intro\">\n      <strong>\u6e2c\u9a57\u76ee\u6a19\uff1a<\/strong>\u9a57\u8b49\u4f60\u662f\u5426\u80fd\u5728\u5be6\u969b\u60c5\u5883\u4e2d\u61c9\u7528 AI Eval \u6846\u67b6\u7684\u77e5\u8b58\u3002<br \/>\n      \u5171 5 \u984c\uff0c\u5305\u542b\u60c5\u5883\u984c\u8207\u932f\u8aa4\u8a3a\u65b7\u984c\u3002\n    <\/div>\n\n    <!-- \u60c5\u5883\u984c 1 -->\n    <div class=\"question-adv\" id=\"qAdv1\">\n      <h3>1. \u4f60\u662f\u4e00\u5bb6\u96fb\u5546\u516c\u53f8\u7684 AI \u7522\u54c1\u7d93\u7406\uff0c\u5718\u968a\u525b\u958b\u767c\u5b8c\u4e00\u500b\u5ba2\u670d Agent\u3002\u5728\u6e96\u5099\u4e0a\u7dda\u524d\uff0c\u4f60\u9700\u8981\u5efa\u7acb\u4e00\u500b Eval \u4f86\u6aa2\u6e2c Agent \u56de\u8986\u7684\u8a9e\u6c23\u662f\u5426\u7b26\u5408\u54c1\u724c\u8abf\u6027\u3002 <span class=\"badge scenario\">\u60c5\u5883\u984c<\/span><\/h3>\n      <div class=\"code-block\">\u4f60\u7684 Eval Prompt \u521d\u7248\u5982\u4e0b\uff1a\n\n\u300c\u8acb\u8a55\u4f30\u4ee5\u4e0b\u6587\u5b57\u7684\u8a9e\u6c23\uff0c\u7d66\u51fa 1-5 \u5206\u7684\u5206\u6578\u3002\n1 \u5206\u8868\u793a\u975e\u5e38\u4e0d\u53cb\u5584\uff0c5 \u5206\u8868\u793a\u975e\u5e38\u53cb\u5584\u3002\u300d\n\n\u4f60\u767c\u73fe\u6bcf\u6b21\u57f7\u884c\u7d50\u679c\u7684\u8b8a\u7570\u5f88\u5927\uff0c\u540c\u4e00\u6bb5\u6587\u5b57\u6709\u6642\u7d66 3 \u5206\u6709\u6642\u7d66 5 \u5206\u3002\n\u4f60\u61c9\u8a72\u5982\u4f55\u6539\u5584\u9019\u500b Eval\uff1f<\/div>\n      <ul class=\"options-adv\">\n        <li onclick=\"checkAnswerAdv(1, 'A', 'C')\">A. \u628a\u6a21\u578b\u63db\u6210\u66f4\u5f37\u7684\u7248\u672c\uff08\u5982\u5f9e GPT-4o Mini \u5347\u7d1a\u5230 GPT-4o\uff09\uff0c\u56e0\u70ba\u66f4\u5f37\u7684\u6a21\u578b\u6253\u5206\u66f4\u6e96<\/li>\n        <li onclick=\"checkAnswerAdv(1, 'B', 'C')\">B. \u589e\u52a0\u66f4\u591a\u6e2c\u8a66\u7bc4\u4f8b\u6578\u91cf\uff0c\u7528\u7d71\u8a08\u5e73\u5747\u4f86\u6d88\u9664\u8b8a\u7570<\/li>\n        <li onclick=\"checkAnswerAdv(1, 'C', 'C')\">C. \u5c07\u6578\u5b57\u5206\u6578\u6539\u70ba\u6587\u5b57\u6a19\u7c64\uff08\u5982\u300cfriendly \/ robotic\u300d\uff09\uff0c\u4e26\u5728 Prompt \u4e2d\u63d0\u4f9b\u5177\u9ad4\u5b9a\u7fa9\u548c Few-Shot \u7bc4\u4f8b<\/li>\n        <li onclick=\"checkAnswerAdv(1, 'D', 'C')\">D. \u76f4\u63a5\u8b93\u4eba\u5de5\u5ba2\u670d\u9010\u4e00\u5be9\u67e5\u6240\u6709 Agent \u56de\u8986\uff0c\u653e\u68c4\u81ea\u52d5\u5316 Eval<\/li>\n      <\/ul>\n      <div class=\"feedback-adv\" id=\"fbAdv1\"><\/div>\n    <\/div>\n\n    <!-- \u60c5\u5883\u984c 2 -->\n    <div class=\"question-adv\" id=\"qAdv2\">\n      <h3>2. \u4f60\u8ca0\u8cac\u7684 AI \u65c5\u884c\u898f\u5283 Agent \u5df2\u7d93\u7528 LLM as a Judge \u9032\u884c\u4e86\u53cb\u5584\u5ea6\u8a55\u4f30\uff0c\u7d50\u679c\u986f\u793a 100% \u7684\u56de\u8986\u90fd\u662f\u300cfriendly\u300d\u3002\u4f46\u4f60\u624b\u52d5\u6aa2\u67e5\u4e86 12 \u500b\u7bc4\u4f8b\u5f8c\uff0c\u767c\u73fe\u5176\u4e2d\u6709 8 \u500b\u56de\u8986\u5176\u5be6\u5f88\u5197\u9577\u3001\u8a9e\u6c23\u6a5f\u68b0\u5316\uff0c\u4f60\u8a8d\u70ba\u61c9\u8a72\u88ab\u6a19\u70ba\u300crobotic\u300d\u3002\u4e0b\u4e00\u6b65\u4f60\u61c9\u8a72\u600e\u9ebc\u505a\uff1f <span class=\"badge scenario\">\u60c5\u5883\u984c<\/span><\/h3>\n      <div class=\"code-block\">\u73fe\u72c0\uff1a\n&#8211; LLM as a Judge \u53cb\u5584\u5ea6\u8a55\u4f30\uff1a12\/12 \u70ba friendly\uff08100%\uff09\n&#8211; \u4f60\u7684\u4eba\u5de5\u6a19\u8a3b\u7d50\u679c\uff1a4\/12 \u70ba friendly\uff0c8\/12 \u70ba robotic\n&#8211; \u5169\u8005\u7684\u5339\u914d\u7387\u6975\u4f4e<\/div>\n      <ul class=\"options-adv\">\n        <li onclick=\"checkAnswerAdv(2, 'A', 'B')\">A. \u76f4\u63a5\u6368\u68c4 LLM as a Judge\uff0c\u6539\u7528\u7d14\u4eba\u5de5\u6a19\u8a3b\u4f5c\u70ba\u552f\u4e00\u8a55\u4f30\u65b9\u5f0f<\/li>\n        <li onclick=\"checkAnswerAdv(2, 'B', 'B')\">B. \u5c07\u4eba\u5de5\u6a19\u7c64\u4f5c\u70ba\u57fa\u6e96\uff0c\u56de\u5230 Eval Prompt \u4e2d\u52a0\u5165\u66f4\u56b4\u683c\u7684\u300cfriendly\u300d\u5b9a\u7fa9\u548c Few-Shot \u7bc4\u4f8b\uff0c\u91cd\u65b0\u57f7\u884c Eval \u4e26\u6bd4\u8f03\u5339\u914d\u7387<\/li>\n        <li onclick=\"checkAnswerAdv(2, 'C', 'B')\">C. \u589e\u52a0\u66f4\u591a LLM Judge \u6a21\u578b\uff08\u5982\u540c\u6642\u7528 GPT-4o \u548c Claude\uff09\uff0c\u53d6\u591a\u6578\u6c7a\u4f5c\u70ba\u6700\u7d42\u6a19\u7c64<\/li>\n        <li onclick=\"checkAnswerAdv(2, 'D', 'B')\">D. \u8a8d\u5b9a\u662f\u4eba\u5de5\u6a19\u8a3b\u7684\u6a19\u6e96\u592a\u56b4\u683c\uff0c\u8abf\u6574\u4eba\u5de5\u6a19\u7c64\u4ee5\u5c0d\u9f4a LLM \u7684\u5224\u65b7\u7d50\u679c<\/li>\n      <\/ul>\n      <div class=\"feedback-adv\" id=\"fbAdv2\"><\/div>\n    <\/div>\n\n    <!-- \u60c5\u5883\u984c 3 -->\n    <div class=\"question-adv\" id=\"qAdv3\">\n      <h3>3. \u4f60\u7684 AI \u7522\u54c1\u5df2\u4e0a\u7dda\u4e09\u500b\u6708\uff0cEval \u7cfb\u7d71\u904b\u4f5c\u7a69\u5b9a\u3002\u6700\u8fd1\u5ba2\u6236\u56de\u5831\u4e00\u985e\u65b0\u554f\u984c\uff1aAgent \u5728\u8655\u7406\u300c\u9000\u6b3e + \u63db\u8ca8\u540c\u6642\u8acb\u6c42\u300d\u6642\u7d93\u5e38\u7d66\u51fa\u77db\u76fe\u7684\u5efa\u8b70\u3002\u4f60\u7684\u73fe\u6709 Eval \u8cc7\u6599\u96c6\u4e2d\u6c92\u6709\u9019\u985e\u6848\u4f8b\u3002\u6700\u5408\u9069\u7684\u8655\u7406\u65b9\u5f0f\u662f\u4ec0\u9ebc\uff1f <span class=\"badge scenario\">\u60c5\u5883\u984c<\/span><\/h3>\n      <div class=\"code-block\">\u73fe\u6709 Eval \u8cc7\u6599\u96c6\uff1a200 \u500b\u7bc4\u4f8b\n\u6db5\u84cb\u5834\u666f\uff1a\u4e00\u822c\u8aee\u8a62\u3001\u55ae\u7d14\u9000\u6b3e\u3001\u55ae\u7d14\u63db\u8ca8\u3001\u8a02\u55ae\u67e5\u8a62\n\u7f3a\u5931\u5834\u666f\uff1a\u9000\u6b3e + \u63db\u8ca8\u8907\u5408\u8acb\u6c42\n\u5ba2\u6236\u56de\u5831\u7684\u554f\u984c\u6848\u4f8b\uff1a15 \u7b46<\/div>\n      <ul class=\"options-adv\">\n        <li onclick=\"checkAnswerAdv(3, 'A', 'D')\">A. \u7acb\u523b\u91cd\u65b0\u8a13\u7df4\u6574\u500b Eval \u6a21\u578b\uff0c\u7528 200 + 15 = 215 \u500b\u7bc4\u4f8b\u5168\u90e8\u91cd\u8dd1<\/li>\n        <li onclick=\"checkAnswerAdv(3, 'B', 'D')\">B. \u66ab\u6642\u95dc\u9589 Agent \u7684\u9000\u6b3e\u529f\u80fd\uff0c\u7b49 Eval \u5b8c\u5584\u5f8c\u518d\u91cd\u65b0\u4e0a\u7dda<\/li>\n        <li onclick=\"checkAnswerAdv(3, 'C', 'D')\">C. \u5c07 15 \u7b46\u6848\u4f8b\u6a19\u8a18\u70ba\u7570\u5e38\u503c\uff0c\u4e0d\u7d0d\u5165 Eval \u8cc7\u6599\u96c6\uff0c\u56e0\u70ba\u6a23\u672c\u592a\u5c11\u4e0d\u5177\u7d71\u8a08\u610f\u7fa9<\/li>\n        <li onclick=\"checkAnswerAdv(3, 'D', 'D')\">D. \u5c07\u9019 15 \u7b46\u554f\u984c\u6848\u4f8b\u52a0\u5165 Eval \u8cc7\u6599\u96c6\u4f5c\u70ba\u300c\u56f0\u96e3\u7bc4\u4f8b\u300d\uff0c\u91dd\u5c0d\u8907\u5408\u8acb\u6c42\u5834\u666f\u64b0\u5beb\u65b0\u7684 Eval\uff0c\u4e26\u6301\u7e8c\u5f9e\u751f\u7522\u6578\u64da\u4e2d\u6536\u96c6\u985e\u4f3c\u6848\u4f8b<\/li>\n      <\/ul>\n      <div class=\"feedback-adv\" id=\"fbAdv3\"><\/div>\n    <\/div>\n\n    <!-- \u932f\u8aa4\u8a3a\u65b7\u984c 1 -->\n    <div class=\"question-adv\" id=\"qAdv4\">\n      <h3>4. \u4e00\u4f4d AI \u7522\u54c1\u7d93\u7406\u8a2d\u8a08\u4e86\u4ee5\u4e0b Eval Prompt \u4f86\u8a55\u4f30\u5ba2\u670d Agent \u7684\u56de\u8986\u54c1\u8cea\uff0c\u4f46\u767c\u73fe\u7d50\u679c\u975e\u5e38\u4e0d\u7a69\u5b9a\u2014\u2014\u540c\u4e00\u6bb5\u56de\u8986\u57f7\u884c\u4e09\u6b21\u5206\u5225\u5f97\u5230 2 \u5206\u30014 \u5206\u548c 5 \u5206\u3002\u9019\u500b Eval \u8a2d\u8a08\u6700\u6839\u672c\u7684\u554f\u984c\u662f\u4ec0\u9ebc\uff1f <span class=\"badge debug\">\u932f\u8aa4\u8a3a\u65b7<\/span><\/h3>\n      <div class=\"code-block\">Eval Prompt\uff1a\n\u300c\u4f60\u662f\u4e00\u500b\u54c1\u8cea\u8a55\u4f30\u5c08\u5bb6\u3002\u8acb\u95b1\u8b80\u4ee5\u4e0b\u5ba2\u670d\u56de\u8986\uff0c\n\u4e26\u6839\u64da\u6574\u9ad4\u54c1\u8cea\u7d66\u51fa 1 \u5230 10 \u5206\u7684\u8a55\u5206\u3002\n\n\u5ba2\u670d\u56de\u8986\uff1a{output}\n\n\u8acb\u76f4\u63a5\u8f38\u51fa\u4e00\u500b\u6578\u5b57\u5206\u6578\u3002\u300d<\/div>\n      <ul class=\"options-adv\">\n        <li onclick=\"checkAnswerAdv(4, 'A', 'B')\">A. \u6c92\u6709\u6307\u5b9a\u4f7f\u7528\u54ea\u500b LLM \u6a21\u578b\u4f86\u57f7\u884c Eval<\/li>\n        <li onclick=\"checkAnswerAdv(4, 'B', 'B')\">B. \u4f7f\u7528\u6578\u5b57\u5206\u6578\u800c\u975e\u6587\u5b57\u6a19\u7c64\u4f5c\u70ba\u8f38\u51fa\uff0cLLM \u8655\u7406\u6578\u503c\u8a55\u5206\u672c\u8cea\u4e0a\u4e0d\u53ef\u9760\uff0c\u61c9\u6539\u7528\u5982\u300cexcellent \/ acceptable \/ poor\u300d\u7b49\u6587\u5b57\u6a19\u7c64<\/li>\n        <li onclick=\"checkAnswerAdv(4, 'C', 'B')\">C. \u8a55\u5206\u7bc4\u570d\u592a\u5927\uff081-10\uff09\uff0c\u61c9\u8a72\u7e2e\u5c0f\u5230 1-5 \u5206\u5c31\u80fd\u89e3\u6c7a\u8b8a\u7570\u554f\u984c<\/li>\n        <li onclick=\"checkAnswerAdv(4, 'D', 'B')\">D. \u7f3a\u5c11\u300c\u4f60\u662f\u54c1\u8cea\u8a55\u4f30\u5c08\u5bb6\u300d\u4ee5\u5916\u7684\u66f4\u8a73\u7d30\u89d2\u8272\u63cf\u8ff0<\/li>\n      <\/ul>\n      <div class=\"feedback-adv\" id=\"fbAdv4\"><\/div>\n    <\/div>\n\n    <!-- \u932f\u8aa4\u8a3a\u65b7\u984c 2 -->\n    <div class=\"question-adv\" id=\"qAdv5\">\n      <h3>5. \u4e00\u500b\u5718\u968a\u5728\u958b\u767c\u591a Agent \u65c5\u884c\u898f\u5283\u7cfb\u7d71\u6642\uff0c\u6309\u7167\u4ee5\u4e0b\u65b9\u5f0f\u9032\u884c\u8a55\u4f30\u3002\u4e0a\u7dda\u5f8c\u537b\u767c\u73fe\u884c\u7a0b\u7d93\u5e38\u8d85\u51fa\u9810\u7b97\u3002\u6839\u64da\u4ed6\u5011\u7684\u8a55\u4f30\u6d41\u7a0b\uff0c\u6700\u53ef\u80fd\u907a\u6f0f\u4e86\u4ec0\u9ebc\uff1f <span class=\"badge debug\">\u932f\u8aa4\u8a3a\u65b7<\/span><\/h3>\n      <div class=\"code-block\">\u7cfb\u7d71\u67b6\u69cb\uff1a\n  \u7814\u7a76 Agent \u2192 \u884c\u7a0b Agent \u2192 \u6700\u7d42\u8f38\u51fa\n  \u9810\u7b97 Agent \u2197\n  \u5728\u5730\u9ad4\u9a57 Agent \u2197\n\n\u8a55\u4f30\u65b9\u5f0f\uff1a\n  \u2705 \u5c0d\u300c\u884c\u7a0b Agent\u300d\u7684\u6700\u7d42\u8f38\u51fa\u9032\u884c\u53cb\u5584\u5ea6 Eval\n  \u2705 \u5c0d\u300c\u884c\u7a0b Agent\u300d\u7684\u6700\u7d42\u8f38\u51fa\u9032\u884c\u5b8c\u6574\u6027 Eval\n  \u2705 \u5728 Prompt Playground \u4e2d\u8fed\u4ee3\u884c\u7a0b Agent \u7684 Prompt\n  \u274c \u672a\u5c0d\u300c\u9810\u7b97 Agent\u300d\u7684\u8f38\u51fa\u55ae\u7368\u9032\u884c\u8a55\u4f30\n  \u274c \u672a\u6e2c\u8a66\u4e0a\u6e38 Agent \u8b8a\u66f4\u5c0d\u4e0b\u6e38\u7684\u5f71\u97ff<\/div>\n      <ul class=\"options-adv\">\n        <li onclick=\"checkAnswerAdv(5, 'A', 'C')\">A. \u884c\u7a0b Agent \u7684 Prompt \u4e0d\u5920\u8a73\u7d30\uff0c\u9700\u8981\u52a0\u5165\u66f4\u660e\u78ba\u7684\u9810\u7b97\u9650\u5236\u6307\u4ee4<\/li>\n        <li onclick=\"checkAnswerAdv(5, 'B', 'C')\">B. \u7f3a\u5c11\u4eba\u5de5\u6a19\u8a3b\u6d41\u7a0b\uff0c\u61c9\u8a72\u8b93\u4eba\u985e\u9010\u4e00\u6aa2\u67e5\u6bcf\u500b\u884c\u7a0b\u7684\u9810\u7b97\u8a08\u7b97<\/li>\n        <li onclick=\"checkAnswerAdv(5, 'C', 'C')\">C. \u53ea\u8a55\u4f30\u4e86\u6700\u7d42\u8f38\u51fa\u7684\u884c\u7a0b Agent\uff0c\u6c92\u6709\u5c0d\u5404\u500b\u5b50 Agent\uff08\u7279\u5225\u662f\u9810\u7b97 Agent\uff09\u9032\u884c\u7368\u7acb\u7684\u5206\u6bb5\u8a55\u4f30\uff0c\u4e5f\u672a\u6e2c\u8a66 Agent \u9593\u7684\u9023\u9396\u5f71\u97ff<\/li>\n        <li onclick=\"checkAnswerAdv(5, 'D', 'C')\">D. \u4f7f\u7528\u7684 LLM \u6a21\u578b\u6578\u5b78\u80fd\u529b\u4e0d\u8db3\uff0c\u9700\u8981\u63db\u7528\u6578\u5b78\u63a8\u7406\u80fd\u529b\u66f4\u5f37\u7684\u5c08\u7528\u6a21\u578b<\/li>\n      <\/ul>\n      <div class=\"feedback-adv\" id=\"fbAdv5\"><\/div>\n    <\/div>\n\n    <div class=\"score-adv\" id=\"scoreAdv\" style=\"display:none;\"><\/div>\n<script>\n\n      let answeredAdv = 0;\n      let correctAdv = 0;\n      const totalAdv = 5;\n      const explanationsAdv = {\n        1: \"\u6839\u64da\u5f71\u7247\u4e2d Aman \u7684\u8aaa\u660e\uff0c\u5373\u4f7f\u662f PhD \u7b49\u7d1a\u7684 LLM \u5728\u8655\u7406\u6578\u5b57\u5206\u6578\u6642\u4ecd\u7136\u975e\u5e38\u4e0d\u53ef\u9760\uff0c\u56e0\u70ba\u9019\u8207 Token \u7684\u8868\u793a\u65b9\u5f0f\u6709\u95dc\u3002\u6b63\u78ba\u505a\u6cd5\u662f\u4f7f\u7528\u6587\u5b57\u6a19\u7c64\uff08\u5982 friendly \/ robotic\uff09\uff0c\u4e26\u63d0\u4f9b\u660e\u78ba\u7684\u5b9a\u7fa9\u548c Few-Shot \u7bc4\u4f8b\u4f86\u6e1b\u5c11\u8b8a\u7570\u3002\u9078\u9805 A \u63db\u6a21\u578b\u4e0d\u80fd\u89e3\u6c7a\u6578\u5b57\u5206\u6578\u7684\u6839\u672c\u554f\u984c\uff1b\u9078\u9805 B \u589e\u52a0\u6a23\u672c\u91cf\u53ea\u662f\u63a9\u84cb\u554f\u984c\u800c\u975e\u4fee\u5fa9\uff1b\u9078\u9805 D \u5b8c\u5168\u653e\u68c4\u81ea\u52d5\u5316\u4e0d\u5177\u898f\u6a21\u5316\u53ef\u80fd\u3002\",\n        2: \"\u5f71\u7247\u4e2d\u5c55\u793a\u7684\u5b8c\u6574\u5de5\u4f5c\u6d41\u7a0b\u6b63\u662f\u9019\u500b\u60c5\u5883\uff1a\u7576 LLM Judge \u7684\u7d50\u679c\u8207\u4eba\u985e\u6a19\u7c64\u4e0d\u5339\u914d\u6642\uff0c\u61c9\u8a72\u4ee5\u4eba\u985e\u6a19\u7c64\u70ba\u57fa\u6e96\uff08ground truth\uff09\uff0c\u7136\u5f8c\u56de\u5230 Eval Prompt \u9032\u884c\u8fed\u4ee3\u2014\u2014\u52a0\u5165\u66f4\u56b4\u683c\u7684\u5b9a\u7fa9\u3001Few-Shot \u7bc4\u4f8b\uff0c\u91cd\u65b0\u57f7\u884c\u4e26\u6bd4\u8f03\u5339\u914d\u7387\u3002\u9078\u9805 A \u6368\u68c4\u81ea\u52d5\u5316\u8a55\u4f30\u7121\u6cd5\u898f\u6a21\u5316\uff1b\u9078\u9805 C \u591a\u6a21\u578b\u6295\u7968\u662f\u9032\u968e\u65b9\u6cd5\u4f46\u4e0d\u662f\u6839\u672c\u89e3\u6c7a\u65b9\u6848\uff1b\u9078\u9805 D \u53cd\u904e\u4f86\u8abf\u6574\u4eba\u985e\u6a19\u6e96\u9055\u80cc\u4e86\u300c\u4eba\u985e\u4f5c\u70ba\u57fa\u6e96\u300d\u7684\u6838\u5fc3\u539f\u5247\u3002Aman \u5f37\u8abf\uff1a\u4f60\u9700\u8981 Eval \u4f86\u8a55\u4f30\u4f60\u7684 Eval\uff0c\u800c\u4eba\u985e\u6a19\u7c64\u5c31\u662f\u90a3\u500b\u9328\u9ede\u3002\",\n        3: \"\u9019\u6b63\u662f\u5f71\u7247\u4e2d\u81ea\u99d5\u8eca\u6bd4\u55bb\u7684\u6838\u5fc3\u89c0\u9ede\uff1a\u4f60\u7121\u6cd5\u4e8b\u5148\u9810\u898b\u6240\u6709\u56f0\u96e3\u5834\u666f\uff0c\u53ea\u6709\u5728\u5be6\u969b\u9047\u5230\u6642\u624d\u6703\u767c\u73fe\u3002\u6b63\u78ba\u505a\u6cd5\u662f\u5c07\u9019\u4e9b\u300c\u56f0\u96e3\u7bc4\u4f8b\u300d\uff08hard examples\uff09\u52a0\u5165 Eval \u8cc7\u6599\u96c6\uff0c\u5efa\u7acb\u91dd\u5c0d\u65b0\u5834\u666f\u7684 Eval\uff0c\u4e26\u6301\u7e8c\u5f9e\u751f\u7522\u6578\u64da\u4e2d\u6536\u96c6\u985e\u4f3c\u6848\u4f8b\u3002\u9078\u9805 A \u5168\u90e8\u91cd\u8dd1\u4e0d\u5fc5\u8981\u4e14\u6c92\u6709\u91dd\u5c0d\u6027\uff1b\u9078\u9805 B \u904e\u5ea6\u53cd\u61c9\uff1b\u9078\u9805 C \u5ffd\u8996\u4e86\u771f\u5be6\u7528\u6236\u56de\u5831\u7684\u554f\u984c\uff0c\u800c\u9019\u4e9b\u6070\u6070\u662f\u6700\u6709\u50f9\u503c\u7684\u6539\u5584\u4fe1\u865f\u3002\",\n        4: \"\u5f71\u7247\u4e2d\u660e\u78ba\u6307\u51fa\uff0cLLM \u5728\u8655\u7406\u6578\u5b57\u5206\u6578\u6642\u672c\u8cea\u4e0a\u4e0d\u53ef\u9760\uff0c\u9019\u662f\u7531 Token \u7684\u8868\u793a\u65b9\u5f0f\u6c7a\u5b9a\u7684\u3002Arize \u7684\u7814\u7a76\u8b49\u5be6\uff0c\u4f7f\u7528\u6587\u5b57\u6a19\u7c64\uff08\u5982 excellent \/ acceptable \/ poor\uff09\u518d\u6620\u5c04\u5230\u5206\u6578\uff0c\u6bd4\u76f4\u63a5\u8b93 LLM \u8f38\u51fa\u6578\u5b57\u8981\u53ef\u9760\u5f97\u591a\u3002\u9078\u9805 C \u7e2e\u5c0f\u7bc4\u570d\u4e26\u4e0d\u80fd\u89e3\u6c7a LLM \u5c0d\u6578\u5b57\u8655\u7406\u4e0d\u7a69\u5b9a\u7684\u6839\u672c\u554f\u984c\uff1b\u9078\u9805 A \u548c D \u96d6\u7136\u4e5f\u662f\u6539\u5584\u65b9\u5411\uff0c\u4f46\u4e0d\u662f\u5c0e\u81f4\u7d50\u679c\u9ad8\u5ea6\u4e0d\u7a69\u5b9a\u7684\u6839\u672c\u539f\u56e0\u3002\",\n        5: \"\u5f71\u7247\u4e2d\u5f37\u8abf\u8a55\u4f30 Agent \u7cfb\u7d71\u6642\uff0c\u4e0d\u80fd\u53ea\u770b\u6700\u7d42\u8f38\u51fa\uff0c\u800c\u8981\u5c0d\u5404\u500b\u5b50\u5143\u4ef6\u9032\u884c\u7368\u7acb\u8a55\u4f30\uff08decompose and evaluate parts of your stack\uff09\u3002\u9019\u500b\u5718\u968a\u53ea\u8a55\u4f30\u4e86\u884c\u7a0b Agent \u7684\u8f38\u51fa\uff0c\u537b\u6c92\u6709\u55ae\u7368\u9a57\u8b49\u9810\u7b97 Agent \u662f\u5426\u6b63\u78ba\u8a08\u7b97\u4e86\u8cbb\u7528\uff0c\u4e5f\u6c92\u6709\u6e2c\u8a66\u4e0a\u6e38 Agent \u8b8a\u66f4\u5c0d\u4e0b\u6e38\u7684\u5f71\u97ff\uff08prompt chaining \u7684\u554f\u984c\uff09\u3002\u9078\u9805 A \u53ea\u95dc\u6ce8 Prompt \u5c64\u9762\u800c\u5ffd\u7565\u4e86\u7cfb\u7d71\u67b6\u69cb\u554f\u984c\uff1b\u9078\u9805 B \u4eba\u5de5\u9010\u4e00\u6aa2\u67e5\u4e0d\u5177\u898f\u6a21\u5316\u80fd\u529b\uff1b\u9078\u9805 D \u63db\u6a21\u578b\u662f\u8868\u9762\u89e3\u6cd5\uff0c\u6839\u672c\u554f\u984c\u5728\u65bc\u7f3a\u5c11\u5206\u6bb5\u8a55\u4f30\u3002\"\n      };\n\n      function checkAnswerAdv(qNum, selected, correctAns) {\n        const question = document.getElementById(\"qAdv\" + qNum);\n        if (question.classList.contains(\"answered\")) return;\n\n        question.classList.add(\"answered\");\n        answeredAdv++;\n\n        const options = question.querySelectorAll(\".options-adv li\");\n        options.forEach((opt) => {\n          opt.classList.add(\"disabled\");\n          if (opt.textContent.startsWith(correctAns + \".\")) {\n            opt.classList.add(\"show-answer\");\n          }\n        });\n\n        const feedback = document.getElementById(\"fbAdv\" + qNum);\n        if (selected === correctAns) {\n          correctAdv++;\n          feedback.className = \"feedback-adv correct\";\n          feedback.innerHTML = \"\u2713 \u6b63\u78ba\uff01\" + explanationsAdv[qNum];\n        } else {\n          feedback.className = \"feedback-adv wrong\";\n          feedback.innerHTML =\n            \"\u2717 \u932f\u8aa4\u3002\u6b63\u78ba\u7b54\u6848\u662f \" + correctAns + \"\u3002\" + explanationsAdv[qNum];\n          options.forEach((opt) => {\n            if (opt.textContent.startsWith(selected + \".\")) {\n              opt.classList.add(\"wrong\");\n            }\n          });\n        }\n\n        if (answeredAdv === totalAdv) {\n          const scoreDiv = document.getElementById(\"scoreAdv\");\n          scoreDiv.style.display = \"block\";\n          let msg = \"\";\n          if (correctAdv === totalAdv) msg = \"\ud83c\udfaf \u5b8c\u7f8e\uff01\u4f60\u5df2\u7d93\u6e96\u5099\u597d\u5be6\u6230\u4e86\uff01\";\n          else if (correctAdv >= Math.ceil(totalAdv * 0.8))\n            msg = \"\ud83d\udcaa \u5f88\u68d2\uff01\u518d\u8907\u7fd2\u4e00\u4e0b\u932f\u7684\u984c\u76ee\";\n          else if (correctAdv >= Math.ceil(totalAdv * 0.6))\n            msg = \"\ud83d\udcd6 \u5efa\u8b70\u91cd\u8b80\u76f8\u95dc\u6bb5\u843d\u5f8c\u518d\u8a66\u4e00\u6b21\";\n          else msg = \"\ud83d\udd04 \u5efa\u8b70\u91cd\u65b0\u95b1\u8b80\u6587\u4ef6\uff0c\u7279\u5225\u6ce8\u610f\u64cd\u4f5c\u6d41\u7a0b\";\n          scoreDiv.innerHTML =\n            \"\u6e2c\u9a57\u5b8c\u6210\uff01\u5f97\u5206\uff1a\" + correctAdv + \" \/ \" + totalAdv + \"<br>\" + msg;\n        }\n      }\n    \n<\/script>\n<\/div>\n\n\n<div class=\"wp-like-post__wrapper\" style=\"\"><button type=\"button\" class=\"wp-like-post__button \" style=\"height: 30px; color: #000000;\" data-post-id=\"561\" data-total-likes=\"0\" data-likes-from-user=\"0\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" fill=\"none\" viewBox=\"0 0 24 24\" stroke-width=\"1.5\" stroke=\"currentColor\" width=\"30\" height=\"30\"><path stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M6.633 10.5c.806 0 1.533-.446 2.031-1.08a9.041 9.041 0 012.861-2.4c.723-.384 1.35-.956 1.653-1.715a4.498 4.498 0 00.322-1.672V3a.75.75 0 01.75-.75A2.25 2.25 0 0116.5 4.5c0 1.152-.26 2.243-.723 3.218-.266.558.107 1.282.725 1.282h3.126c1.026 0 1.945.694 2.054 1.715.045.422.068.85.068 1.285a11.95 11.95 0 01-2.649 7.521c-.388.482-.987.729-1.605.729H13.48c-.483 0-.964-.078-1.423-.23l-3.114-1.04a4.501 4.501 0 00-1.423-.23H5.904M14.25 9h2.25M5.904 18.75c.083.205.173.405.27.602.197.4-.078.898-.523.898h-.908c-.889 0-1.713-.518-1.972-1.368a12 12 0 01-.521-3.507c0-1.553.295-3.036.831-4.398C3.387 10.203 4.167 9.75 5 9.75h1.053c.472 0 .745.556.5.96a8.958 8.958 0 00-1.302 4.665c0 1.194.232 2.333.654 3.375z\" \/><\/svg><\/button><div class=\"wp-like-post__count\"\">0<\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>Arize \u7684 AI \u7522\u54c1\u7d93\u7406 Aman Khan \u5728 AI Engineer World&#8217;s F [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":560,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rolpb_likes":0,"footnotes":""},"categories":[495],"tags":[529,527,530,528,510],"class_list":["post-561","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-llm","tag-ai-","tag-arize","tag-llm-as-a-judge","tag-llm-"],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":7}},"_links":{"self":[{"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/posts\/561","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/comments?post=561"}],"version-history":[{"count":0,"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/posts\/561\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/media\/560"}],"wp:attachment":[{"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/media?parent=561"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/categories?post=561"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.itsmygo.uk\/wordpress\/wp-json\/wp\/v2\/tags?post=561"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}