Get Content

Overview#

Bring in content from a public web page or a YouTube video.

Use this block when its responsibility is clear in the graph. Give it a name that describes the work it performs, then test it with representative input before publishing.

Working examples#

These are complete starting points, not decorative diagrams. Every graph is also available in the main template library and is checked by the workflow compiler.

1. Competitor landing-page capture#

You will build: Readable page content for positioning analysis.

Brings public evidence into the workflow before analysis. Use only public pages and inspect the returned source metadata.

Open this template and adapt the labels, prompts, or credentials to your own workflow.

2. Documentation change review#

You will build: A clean documentation snapshot.

Creates a stable input for release-impact analysis. Check the canonical URL and scrape timestamp before relying on the result.

Open this template and adapt the labels, prompts, or credentials to your own workflow.

3. Job description evidence pack#

You will build: Role requirements and company language.

Gives outreach workflows first-party hiring context. Keep the source URL with the output so claims can be verified.

Open this template and adapt the labels, prompts, or credentials to your own workflow.

Ports#

• Input: Link (in): any, optional, single.
• Output: Content (output): any, optional, single. Legacy aliases: out.

Inspector#

• Content source: key source; select; options: web, youtube; default: "web".
• Link source: key sourceMode; select; options: `, run_input; default: ""`.
• Fetch profile: key profile; select; options: static, adapter; default: "static"; Browser rendering is unavailable until isolated request interception can enforce the same egress policy..
• Link: key url; text; default: ""; Example: https://example.com/article.
• Question: key question; textarea; default: "Which link should we use?".
• Description: key description; textarea; default: "Paste one public link.".
• Caption language (optional): key language; text; Example: e.g. en, es, hi.
• Transcript style: key outputFormat; select; options: plain_text, timestamped.
• Primary content: key extract; select; options: markdown, text, structured, html, links, custom; default: "markdown".
• Max pages: key maxPages; number; range: 1 to 20; default: 1.
• Respect robots.txt: key respectRobots; switch; default: true.
• Blocked-page fallback: key fallback; select; options: none, amp, archive; default: "none".
• Timeout (ms): key timeout; number; range: 1000 to 120000; default: 30000.
• Max response bytes: key maxBytes; number; range: 1024 to 5242880; default: 5242880.
• Allowed Hosts (comma-separated): key allowOnly; text; default: [].
• Denied Hosts (comma-separated): key denyHosts; text; default: ["127.0.0.1","localhost","0.0.0.0"].
• Include HTML: key includeHtml; switch; default: false.
• Include app state: key includeAppState; switch; default: false.
• get-content editor: manages selectors, crawl, followPagination with edit, validate, preview support.

Behavior#

• Input requirement: Provide one of ports in or config fields url, sourceMode.
• Execution: Handled by the handler using web-scrape.
• Timeout: 30000 ms by default (1000 to 120000 ms).

Cost#

Uses outbound HTTP transport and may contribute to the run's compute cost.

Security#

• No credentials are used.
• The node reads from an external service without intentionally mutating it.

Retry#

The runtime may retry this node according to the compiled execution policy.

Production checklist#

• Test the happy path and one malformed or empty input.
• Keep credentials out of prompts and published graph configuration.
• Set a timeout and retry policy that matches the operation's side effects.
• Name the final result so a runner can tell what success looks like.

Output Schema#

Top-level fields: url, finalUrl, status, ok, title, description, author, publishedAt, siteName, lang, canonicalUrl, image, content, markdown, text, html, structured, appState, links, images, fields, _meta, pages, complete, warnings, stopReason.

Metadata fields under _meta: strategy, source, extractorUsed, hadStructuredData, blocked, blockReason, robotsAllowed, attempts, bytes, truncated, durationMs, pagesFetched, pagesFailed, archiveSnapshot.