Scott Roland, Verilab Inc, Austin, TX
As a verification engineer, it is common to be given a design to test that is based on earlier design. Presumably, that existing design also comes with a proven verification environment and suite of tests. Unfortunately, the legacy verification environment might be rather rudimentary. If we are going to create a modern, VMM-based, verification environment for the new design, then what can and should be done with the existing “simple” tests? As Joel Spolsky once said, “the single worst strategic mistake [is deciding] to rewrite the code from scratch.”
Verification reuse is just as valuable as design reuse. This can be true even if the legacy tests are a set of text command files that are read in at runtime and perform “dumb” directed testing. If the original tests are of good quality, then they will still cover important functionality of the design. They might also tests critical corner cases or problems that were seen during the initial development of the design being reused. In addition, reusing existing directed tests could help you achieve some early testing of your device faster than if you had started your verification effort from scratch.
The first way you can leverage a legacy testbench is to reuse some of the code responsible for stimulating and monitoring the DUT. As you build the new VMM environment you can properly encapsulate the existing code into the relevant VMM transactors.
Next, it would be nice to reuse the original tests themselves. The tests could be a set of tasks calls or a text file containing commands, as mentioned before. You could translate each test individually into a proper VMM test that generates transaction directly, but it would be better to create an adaption layer between the original tests and the new environment. That would obviate the need to modify the tests and allow the VMM environment to also handle new tests written for the old testbench.
To illustrate an example of using a file-based test in a VMM environment, I chose to extend the memsys_cntrlr example that is distributed with VMM release 1.2.1 in the directory sv/examples/std_lib. The example contains a number of scenarios that are implemented as extensions of the VMM Multi-stream Scenario class. I want to create an additional scenario that reads in a command file and generates transaction based on the file. First, assume that my command file contains lines that specify the command, address and data:
WRITE 8888_8888 2A
READ 2222_2222 24
WRITE 3333_3333 1A
READ 5555_5555 81
READ 7777_7777 42
Using the existing cpu_directed_scenario as a template, I created a new cpu_filebased_scenario. The execute() task, that is responsible for defining what the scenario does, takes care of reading in the test file and calling the proper write/read tasks based on the individual commands. Since the original command file specified the expected return value of every read command, the read task checks the actual return value against the given expected value. Eventually, you might create a reference model that would enable the VMM environment to predict the expected read values. Implementing the directed check in the scenario enables you to run the legacy tests before a reference model is completed and later validate the initial tests and reference model against each other. Here is the implementation of the scenario:
/// Scenario that executes commands read from a directed test file.
class cpu_filebased_scenario extends cpu_rand_scenario;
…
/// Overloaded version of vmm_ms_scenario::execute(). Body of our scenario.
/// Reads each line in the file and performs the specified action.
task execute(ref int n);
integer fileID;
bit [8*5:1] cmd_str;
bit [7:0] data;
bit [31:0] addr;
if (chan == null) chan = get_channel("cpu_chan");
fileID = $fopen("test.file", "r" );
// Lines look like: "CMD ADDRESS DATA"
while ($fscanf(fileID, "%s %h %h", cmd_str, addr, data) != -1) begin
unique case (cmd_str)
"WRITE": this.write(addr, data);
"READ" : this.read (addr, data);
default: `vmm_error(this.log, $psprintf("Unknown command %s", cmd_str));
endcase
n += 1;
end
$fclose(fileID);
endtask
/// Send a write transaction for the given address and data.
task write(input bit [31:0] addr,
input bit [ 7:0] data);
cpu_trans tr = new();
tr.randomize() with {tr.address == addr; tr.kind == WRITE;tr.data == data;};
chan.put(tr);
endtask
/// Send a read transaction for the given address and check for the given expected data.
task read(input bit [31:0] addr,
input bit [ 7:0] exp_data);
cpu_trans tr = new();
tr.randomize() with {tr.address == addr; tr.kind == READ;};
chan.put(tr);
if (tr.data !== exp_data) begin
`vmm_error(this.log, $psprintf("READ(A:%X, D:%X) did not match expected:%X",
addr, tr.data, exp_data));
end
endtask
endclass
After defining the scenario class, I created an extension of vmm_test that tells the VMM factory to use the file-based scenario for this test and run it once. The VMM architecture and factory makes it possible to run the legacy tests just like any randomized VMM tests. Plus, it does not require modifying any other component in the environment. Here is the implementation of the test class:
class test_filebased extends vmm_test;
…
function void configure_test_ph();
// Tell the factory which scenario class to use for this test.
cpu_rand_scenario::override_with_new("@%*:CPU:rand_scn",
cpu_filebased_scenario::this_type(), log, `__FILE__, `__LINE__);
endfunction
function void build_ph();
// Run the scenario only once.
vmm_opts::set_int("%*:num_scenarios", 1);
endfunction
endclass
Since the directed test file is read into a new environment, it stands to reason that the driver in the new environment could act differently than the one in the legacy environment. For example, the driver might try to combine multiple transactions into bursts or reorder them. While this should probably be done in a specific scenario in the VMM environment, not the driver, you should compare the final stimulus that is performed on the DUT between the two environments. Only after you have done such an assessment can you say that the testcase is being reused for it’s original intent.
Once you have the legacy tests running in a modern VMM environment, you can enhance the environment to have randomization, self-checking and functional coverage. You can analyze the existing tests with a coverage model to determine what new tests you need to write to verify functionality that was initially missed or has been added or modified. The coverage model can also tell you which legacy tests duplicate functionality in other tests, providing justification for getting rid of legacy tests and giving confidence in the quality of your VMM environment.
