Google Cloud Digital Leader: AI Framing and Data Quality

Define an AI task’s input, output, baseline, and evaluation criteria; examine data quality before interpreting an attractive result.

Objective: specify the decision before the model

Use the preceding lesson’s attention to data meaning to turn an AI ambition into a testable task. “Add AI to customer support” is too broad to evaluate. Identify the input the system receives, the output it should produce, the person who uses that output, and what a serious error would look like. Comparing a model with the current process or a simple rule helps identify whether the proposed system improves the actual task. This framing method is an original teaching tool, not a promise of a particular model’s performance.

Different outputs require different evidence

A prediction from historical business data and a generated explanation solve different problems. BigQuery’s documented use cases include predictive analytics; generative systems can produce responses whose usefulness and factuality require evaluation. A polished paragraph is not evidence that its underlying claim is correct. Likewise, an accurate total can hide poor behavior for an important subset. Decide which output property matters before choosing a metric, reference answer, or human rubric. Inspect both aggregate results and individual examples against the specified task.

Quality begins in the input

Google’s responsible-AI guidance identifies input quality, accuracy, bias, unusual cases, and uneven language representation as limitations that can affect responses. Check whether a field means the same thing across sources, whether a supposedly current record is obsolete, and whether the evaluation examples resemble actual use. Missing or misleading inputs can produce misleading outputs even when the system executes successfully. A larger dataset does not automatically repair a mistaken definition. Record exclusions and uncertainty so a favorable score cannot conceal a known coverage gap.

Worked example

Fictional case: a library wants a tool to draft answers about its borrowing rules. The input is a member’s question plus the current approved policy. The output is a draft answer for a librarian to review. First define success as answering the question consistently with that policy. Second, include tests about renewals, exceptions, and ambiguous questions. Third, test an obsolete policy document and a question missing a necessary date. The appropriate behavior can be to ask for information or refer the case, rather than invent an exception. Finally compare the draft-and-review workflow with the librarian’s current process, including correction effort.

Evaluate the task and the changed data

The Gen AI evaluation service supports datasets, tailored or fixed rubrics, computation-based metrics, and custom evaluation functions. Its documentation recommends assembling examples for the specific use case and interpreting both scores and responses. If the library adds a second language or changes its policy, the previous result does not establish quality for that new condition. Add relevant cases and compare behavior again. Public leaderboards can inform exploration, but they do not substitute for tests of the organization’s own inputs and acceptance criteria.

Transfer check

A repair company receives inconsistent equipment names from two branches and proposes selecting a larger model before investigating errors. What should it inspect first? Check: reconcile or explicitly map the equipment definitions and examine whether the evaluation data represents both branches. A model comparison can follow, using the corrected task and data contract. Retrieve one input-quality failure with the cards, then use the next lesson to distinguish a generated answer from an agent that can take actions through tools.

Try it, then explain it.

Complete a short set without looking at the example. After each answer, explain which evidence or rule supports your decision and why the alternatives do not fit.

Practice 5 questions

Sources & authorship

Original OpenExam explanations and examples. References support the concepts; they are not the source of copied questions. Independent educator review is pending.