Source code
Revision control
Copy as Markdown
Other Tools
Test Info:
/* Any copyright is dedicated to the Public Domain.
"use strict";
// 178 tokens
const tinyArticle =
"\nThe tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building, and the tallest structure in Paris. Its base is square, measuring 125 metres (410 ft) on each side. During its construction, the Eiffel Tower surpassed the Washington Monument to become the tallest man-made structure in the world, a title it held for 41 years until the Chrysler Building in New York City was finished in 1930. It was the first structure to reach a height of 300 metres. Due to the addition of a broadcasting aerial at the top of the tower in 1957, it is now taller than the Chrysler Building by 5.2 metres (17 ft). Excluding transmitters, the Eiffel Tower is the second tallest free-standing structure in France after the Millau Viaduct.\n";
// 286 tokens
const smallArticle =
"\nBank of America says widespread outage is nearly fixed | CNN Business\nMatt Egan\n2-3 minutes\n\nCNN —\n\nBank of America customers reported having trouble accessing their bank accounts on Wednesday, but the bank said the apparent glitch has been largely fixed.\n\nReports about a problem at Bank of America spiked around 12:45 pm ET on Downdetector, which collects data on outages.\n\nMany customers complained they could not see their account balances. Some who could access their accounts were startled to see zero balances.\n\nIn a statement to CNN, Bank of America acknowledged that 'some clients are experiencing an issue accessing their accounts and balance information today.'\n\n'These issues are being addressed and have largely been resolved,' Bank of America said. 'We apologize for any inconvenience.'\n\nBank of America declined to specify what caused the problem. It s not clear when all systems will be fully back to normal.\n\nMultiple CNN employees with Bank of America accounts could not log into their accounts online on Wednesday afternoon. A message displayed for one Bank of America customer said the current balance amount for one or multiple accounts 'may be temporarily unavailable.'\n\n'Five accounts show zero balance, over 20K,' one Bank of America customer on Downdetector wrote.\n\nAnother said he can't log in but his wife can and her accounts display no balance.\n\n'Shows my debt just fine tho,' another user said.\n\nThis story has been updated with additional developments.\n";
// 512 tokens
const mediumArticle =
"\nPush to map Great Lakes bottom gains momentum amid promises effort will help fishing and shipping | CBC News\nThe Associated Press Posted: Oct 02, 2024 10:58 AM EDT | Last Updated: October 2\n7-9 minutes\n\nWindsor\n\nOnly 15 per cent of the lakes' bottom has been mapped, according to the Great Lakes Observing System. The organization has been pushing since 2018 to secure funding to map the entire lakebed in high-resolution.\nOnly 15% of lakes' bottom has been mapped, per Great Lakes Observing System\n\nA sunken vessel in Lake Huron\nThe Africa, a wooden cargo steamer built in 1875, vanished on Lake Huron during an early October snowstorm in 1895. The vessel hadn't been seen in 128 years. (Zach Melnick)\n\nWhat lies beneath the Great Lakes' waves is largely unknown, but there's a new push to learn more about thousands of shipwrecks, underwater infrastructure and the impacts of climate change on the bottom of the world's largest freshwater system.\n\nOnly 15 per cent of the lakes' bottom has been mapped, according to the Great Lakes Observing System (GLOS). The organization has been pushing since 2018 to secure funding to map the entire lakebed in high-resolution.\n\nAdvocates argue the effort would paint a detailed picture of shipwrecks, infrastructure such as pipelines and topography that would inform storm surge models as the climate warms.\n\nJennifer Boehme grew up scouting beaches around her home in St. Petersburg, Fla., for whatever she could find. Rocks, sand dollars, coquina mollusks — anything the ocean gave up.\n\nNow, 40 years later, Boehme wants to launch another treasure hunt. She's the executive director of GLOS.\n\n The 4 factors that have led to a 'golden age' of discovery for Great Lakes shipwrecks\n\n How will Lake Huron's shoreline look in 100 years? Here's the map\n\nThe map, she says, also will help ships avoid submerged hazards, identify fisheries and inform erosion, storm surge and flooding models as climate change intensifies.\n\nOne of the things that keeps me going is the idea of the discovery aspect of it, Boehme said.\n\nThere's a lot we don't know about the lakes. We know more about the surface of the moon.\n\nJennifer Boehme is the executive director for the Great Lakes Observing System.\nJennifer Boehme is the executive director for the Great Lakes Observing System.";
// 1024 tokens
const bigArticle =
"The Transformative Power of Traveling: Exploring the World and Ourselves\nTraveling is much more than moving from one location to another. It’s a transformative journey that reshapes perspectives, deepens understanding, and rejuvenates the spirit. Whether it's a weekend road trip, a months-long backpacking adventure, or an exploration of cultural hubs across the globe, traveling has the potential to profoundly impact the way we view the world and ourselves.\n\nThe Thrill of Discovery\nAt its core, traveling is about discovery. For some, it’s the thrill of stepping into a bustling city like Tokyo, Paris, or Istanbul, where every street offers something new: food, art, and traditions woven into daily life. For others, it's the serenity of nature that beckons — trekking through the Himalayas, wandering across the Saharas golden dunes, or diving into the vibrant coral reefs of the Great Barrier Reef. Each journey unravels layers of wonder, revealing a world far more diverse and intricate than we imagine.\n\nBut it’s not just external landscapes we explore; traveling also encourages inner discovery. When we step out of our comfort zones, we confront unfamiliar situations that test resilience, adaptability, and creativity. Lost in an unfamiliar city without Wi-Fi? You might find yourself relying on the kindness of strangers or your resourcefulness. A delayed flight or a sudden change in plans? These moments teach patience and the art of letting go.\n\nBridging Cultures and Building Connections\nTraveling allows us to bridge gaps between cultures, fostering a deeper understanding of humanity's shared values. When we interact with people from different walks of life, we begin to appreciate their customs, beliefs, and way of living. Sharing meals with locals, participating in traditional festivals, or simply engaging in conversation over coffee broadens horizons and builds empathy.\n\nConsider a traveler visiting Morocco, where the vibrant souks of Marrakech provide a sensory overload of spices, textiles, and handmade crafts. Beyond the market’s hustle, they might join a family for a traditional tagine dinner, learning not just about the dish but also about Moroccan hospitality. These moments linger in memory, reminding us of our shared human experience, no matter where we’re from.\n\nLanguage barriers often turn into opportunities for connection. A smile, a gesture, or a shared laugh transcends words. These interactions remind us that kindness and curiosity are universal languages, opening doors to friendships and enriching our lives.\n\nEscaping the Routine\nIn our fast-paced world, filled with deadlines, notifications, and responsibilities, traveling offers a much-needed escape from the daily grind. It disrupts monotony, encouraging a sense of spontaneity and adventure. Whether it’s waking up to the call of exotic birds in a rainforest or watching the sun set over a tranquil beach, these moments ground us, reminding us of life’s simple yet profound joys.\n\nThe journey itself becomes an opportunity to practice mindfulness. Without the usual demands of work or routines, travelers can immerse themselves in the present moment. Sipping coffee in a quaint Italian piazza, marveling at the Northern Lights in Iceland, or walking through Kyoto’s serene bamboo forests invites us to slow down and savor life.\n\nTraveling Responsibly\nAs we venture into the world, it’s crucial to acknowledge our impact on the places we visit. Overtourism, environmental degradation, and cultural exploitation are real challenges. Traveling responsibly means making choices that respect local communities, preserve natural resources, and minimize harm.\n\nOpting for eco-friendly accommodations, supporting local businesses, and learning about the cultural etiquette of a destination are small but impactful steps. Traveling with intention — valuing quality experiences over quantity — ensures that we leave a positive footprint. It's about giving back to the places that give so much to us.\n\nThe Lessons We Bring Home\nPerhaps the most remarkable thing about traveling is the way it changes us. Every trip, no matter how short or long, leaves a mark. We return home not just with souvenirs but with stories, insights, and often a renewed sense of gratitude.\n\nTraveling teaches humility. Standing before the vastness of the Grand Canyon or the towering spires of a medieval cathedral reminds us of our small place in the grand scheme of things. It also instills confidence, as navigating foreign streets or overcoming travel hiccups proves our capability.\n\nFinally, traveling ignites a sense of curiosity that lingers long after the journey ends. It encourages us to keep exploring, whether it's the next destination on our bucket list or the unexplored corners of our own neighborhoods. This mindset of discovery enriches everyday life, reminding us that the world is filled with opportunities for awe.\n\nConclusion\nTraveling is not just a luxury or a pastime; it's an investment in personal growth and global understanding. It challenges us to step outside our routines, confront the unfamiliar, and embrace the beauty of diversity. In doing so, we find not only the magic of the world but also the boundless potential within ourselves.\n\nSo pack your bags, set out with an open heart, and let the world teach you. Because, as the saying goes, Travel is the only thing you buy that makes you richer.";
let testData = [];
const distilBartModel = {
taskName: "summarization",
modelId: "Xenova/distilbart-cnn-12-6",
dtype: "q8",
};
const qwenModel = {
taskName: "text-generation",
modelId: "onnx-community/Qwen2.5-0.5B-Instruct",
dtype: "q8",
};
const articles = [
{ data: tinyArticle, type: "tiny" },
{ data: smallArticle, type: "small" },
{ data: mediumArticle, type: "medium" },
{ data: bigArticle, type: "big" },
];
let numEngines = 0;
for (const model of [distilBartModel, qwenModel]) {
for (const article of articles) {
// Replace all non-alphabnumeric or dash or underscore by underscore
const perfName = `${model.modelId.replace(/\//g, "-")}_${article.type}`;
const engineId = `engine-${numEngines}`;
const options = { ...model, article: article.data, engineId, perfName };
numEngines += 1;
testData.push(options);
}
}
const perfMetadata = {
owner: "GenAI Team",
name: "ML Summarizer Model",
description: "Template test for latency for Summarizer model",
options: {
default: {
perfherder: true,
perfherder_metrics: [
{
name: "latency",
unit: "ms",
shouldAlert: true,
},
{
name: "memory",
unit: "MB",
shouldAlert: true,
},
],
verbose: true,
manifest: "perftest.toml",
manifest_flavor: "browser-chrome",
try_platform: ["linux", "mac", "win"],
},
},
};
requestLongerTimeout(120);
// To run locally
// pip install huggingface-hub
// huggingface-cli download {model_id} --local-dir MOZ_FETCHES_DIR/onnx-models/{model_id}/{revision}
// Update your test in
// Then run: ./mach lint -l perfdocs --fix
// This will auto-generate docs
async function run_summarizer_with_perf({
taskName,
modelId,
article,
dtype,
engineId,
perfName,
}) {
let chatInput = article;
const minNewTokens = 195;
const maxNewTokens = 200;
let requestOptions = {
max_new_tokens: minNewTokens,
min_new_tokens: maxNewTokens,
};
const options = new PipelineOptions({
engineId,
taskName,
modelHubUrlTemplate: "{model}/{revision}",
modelId,
modelRevision: "main",
dtype,
});
if (taskName === "text-generation") {
chatInput = [
{
role: "system",
content:
"Your role is to summarize the provided content as succinctly as possible while retaining the most important information",
},
{
role: "user",
content: article,
},
];
requestOptions = {
max_new_tokens: minNewTokens,
min_new_tokens: maxNewTokens,
return_full_text: true,
return_tensors: false,
do_sample: false,
};
}
const request = {
args: [chatInput],
options: requestOptions,
};
await perfTest(`sum-${perfName}`, options, request);
}
add_task(async function test_ml_distilbart_tiny_article() {
await run_summarizer_with_perf(testData[0]);
});
add_task(async function test_ml_distilbart_small_article() {
await run_summarizer_with_perf(testData[1]);
});
add_task(async function test_ml_distilbart_medium_article() {
await run_summarizer_with_perf(testData[2]);
});
add_task(async function test_ml_distilbart_big_article() {
await run_summarizer_with_perf(testData[3]);
});
add_task(async function test_ml_qwen_small_article() {
await run_summarizer_with_perf(testData[4]);
});
add_task(async function test_ml_qwen_medium_article() {
await run_summarizer_with_perf(testData[5]);
});
add_task(async function test_ml_qwen_medium_article() {
await run_summarizer_with_perf(testData[6]);
});
add_task(async function test_ml_qwen_big_article() {
await run_summarizer_with_perf(testData[7]);
});