( ESNUG 467 Item 9 ) -------------------------------------------- [07/26/07]
From: Stu Hecht <stuh=user domain=broadcom bot calm>
Subject: The 14 gotchas we found while using the new PrimeTime DMSA tool
Hi John,
Here's the PrimeTime DMSA user experience that I promised you.
For those who don't know, PrimeTime Distributed Multi-Scenario Analysis
(DMSA) allows you to analyze all mode/corner combinations at once. So, if
you have 2 functional modes plus scan shift plus scan capture (4 modes) and
analyze in max and min corners (2 corners) you'll have 8 (4 * 2) scenarios.
First you load this up in PrimeTime using up to 9 machines and 8 licenses
(the master uses a machine but not a license). Then you do report_timing
to get the worst timing path across all scenarios.
It's pretty cool to be able to see the single worst timing path for the
entire chip with one command instead of having to type the same command into
many PT shells! It gets even better when you start to do ECOs (size_cell,
insert_buffer, ...) because you don't have to wonder if you hurt setup when
you fixed a hold problem because you can see all the timing at once. Even
better, you can run a script to fix all the hold problems and know that it
won't break setup (the script is available from Synopsys). If you have
enough machines and licenses then DMSA runs as quickly as a single PrimeTime
session so using DMSA doesn't slow you down.
What not to do:
We were using DMSA for the first time and thought that we should apply all
our constraints using DMSA, do scenario-specific reporting from within DMSA,
save sessions from within DMSA, etc. That turned out to be a big mistake.
After trying this and talking to Synopsys it seems that a better use model
is to create all the scenario saved sessions using non-DMSA mode and then
loading all the saved sessions into DMSA and using DMSA just for the work
that needs all the scenarios loaded.
Why? Because there are some disadvantages to DMSA and you don't want to be
saddled with them if you don't need DMSA. For example, DMSA runs all the
scenarios in lock step. When you run the scenarios separately they can
start running as resources (CPUs and licenses) become available. For DMSA
you need to decide ahead of time how many machines to use (and then wait for
that number to become available) and if DMSA can't get a license for each
machine then it will happily shuffle licenses around but that will leave
machines sitting idle waiting for a license. Unfortunately DMSA only tries
to acquire licenses once; it can't get more licenses as they become
available.
Also, DMSA doesn't let the slaves do a save session. So if some scenarios
finish quickly the session doesn't get saved until all the scenarios
complete. That means you can't load the session into an interactive PT
session until everything is done (the bosses didn't like us telling them
that as we were trying to tape-out).
It also means that if any one scenario dies that none of the sessions get
saved. This happened to us a few times and we had to run all scenarios
again -- what a waste of licenses/machines plus we had no saved sessions
to look at!
Another drawback to using DMSA when you don't really need it is that
licenses and machines are in use for all scenarios until the slowest one
completes. Our scan scenarios ran pretty quickly but then sat idle for a
long time just wasting resources.
A strange DMSA issue that we saw is that the master became our bottleneck
(even though all it did was send a single command to the slaves to execute
a script). We think this was because all the STDOUT from all the slaves
needed to get processed by the master and our STDOUT was huge (later reduced
by suppressing some messages). We had a lot of scenarios (not just 8) and
the next revision of the chip will have even more scenarios so all of these
problems would just get worse.
Now we just use Perl & make to do the initial PT runs to create the saved
sessions. Once all the sessions are created we run DMSA, load the sessions
and just do some reporting. We do need to get resources twice doing it this
way (once to create the initial save sessions and then again for DMSA
reporting) but only the DMSA part needs all the resources at once.
DMSA scripts:
There are two great DMSA scripts you should use. They were both written by
Chris Papademetrious of Synopsys. The first is a small script that helps
you restore all your saved sessions into DMSA:
# restore_dmsa_session.tcl - create scenarios from a
# directory of saved sessions
#
# v1.0 07/13/2006 chrispy
# initial release
proc restore_dmsa_session {args} {
parse_proc_arguments -args $args results
if {[set dirs [glob -nocomplain -type f $results(dir_name)/*/lib_map]]
eq {}}
{
echo "Error: no save_session directories found."
return 0
}
foreach dir $dirs {
set dir [file dirname $dir]
echo "Defining scenario '[set name [file tail $dir]]'."
create_scenario -name $name -image $dir
}
}
define_proc_attributes restore_dmsa_session \
-info "Restores PrimeTime sessions in DMSA" \
-define_args \
{
{dir_name "Dir name to restore from" "dir_name" string required}
}
The second script is a big (but easy to use) script that will fix hold
without breaking setup. It's here on Synopsys SolvNet:
Fixing Hold Violations Across Many Modes and Corners With Distributed
Multi-Scenario Analysis
https://solvnet.synopsys.com/retrieve/018510.html
Hold fixing:
Chris' fix hold script really saved us. As I already mentioned, we had a
lot of scenarios. It would have been very hard to fix all our hold problems
without using DMSA. For layout, we tried to combine all our modes into one
supermode. That worked pretty well but we did get some BlastFusion hold
fixes that broke setup and some hold problems weren't fixed. If we had
infinite time we might have been able to give BlastFusion an accurate
picture of the whole problem. Even then, the PT timing would be somewhat
different and we would still have to do hold fixing from PT. So we had a
bunch of hold problems to fix (not something that could have been done
manually).
Using the DMSA fix hold script we could fix all our hold problems at once.
It fixed scan min corner holds and functional min corner holds (and yes, our
hold problems in max corners as well) all while looking at setup time in all
scenarios so that no new setup problems were introduced.
Yes, DMSA used a lot of licenses and machines to fix hold. But since it
solved a problem that would have been very hard to solve it was well worth
it. It was easy to run so that we could focus on other problems. We were
often asked if we could use a subset of our scenarios to fix hold. There
might be some subset that would work, but I think figuring out the correct
subset would take a while, would not reduce the number of scenarios too much
and would add a big risk of either breaking setup or not fixing all the hold
problems. (If you aren't in a rush to tape-out then fewer machines and/or
licenses can be used at the cost of wall clock time but we needed to
tape-out ASAP.)
ECO things to watch out for:
We did have to iterate a little on hold fixing since PT ECO timing isn't
post-layout accurate. In the future we hope to experiment with the
-setup_margin and -hold_margin options in the fix_hold script to see if we
can close timing quicker without causing any new problems (many more hold
buffers or having hold problems left unfixed).
Getting the ECOs out of PT and into Magma did take a surprising amount of
work (there were some flow issues with write_changes and write_astro_changes
and a script needed to be written to modify the PT output).
Since PT can't write a Magma format ECO list we used write_astro_changes to
get the information we needed into an ASCII file. Then we had some homebrew
script converts it to Magma format. I didn't work on that script so I don't
know the details here.
Since PT can't write out a Verilog netlist, we couldn't do hand edits to a
post-PT ECO netlist to fix functional problems.
More things to watch out for:
As you can see, it is easy to use a lot of PT licenses once you get started
using DMSA. I'm sure Synopsys loves this. But I love this as well since we
get to decide how many licenses to use (sometimes just loading functional
scenarios, sometimes just scan scenarios, sometimes just max corner
scenarios and sometimes just using non-DMSA). Once I have new constraints
written or ECOs done, I can quickly load up a full DMSA session and check
that the constraints or ECOs are OK in all scenarios.
Sure, using a full DMSA session for all the work would be safer, but I have
to share resources and have also found that getting a few licenses/machines
is often much quicker than getting resources for all scenarios.
Another limitation we found is that PT DMSA can only get licenses from a
single license server. Our licenses were split between 2 license servers.
In some cases we found that there were enough total licenses available (8
in the example I've been using) but if one license server had 3 licenses
available and the other had 5 available, we might only be able to get 3
licenses because when the PT master started it got a single license from
the server that had 3 licenses available thus forcing all licenses to get
pulled from that license server.
Ideally it would pull licenses from all available license servers. We have
requested that Synopsys fix this problem.
Some commands are disabled in the DMSA master. This isn't surprising since
any command that accesses the design needs to passed to the slaves. That's
how report_timing works. Hopefully report_constraint, which is currently
disabled, will be supported in a future version. I dream of a day when I
can do a "report_constraint -recalc" from the master (yes, I want path based
analysis, too). I also hope that get_ports, get_cells, get_pins will also
be supported in the master. I use these all the time during debug and
trying to get that information from a slave is annoying (and with lots of
scenarios can produce lots of screen output).
One last warning I need to add here is that PT 2007.06 has a DMSA bug that
get_timing_paths will return a different number of paths each time the
command is executed. I noticed this when I saw the exact same thing in
report_timing. For me, DMSA report_timing always showed the worst path, but
would also sometimes show another path. Synopsys is looking into this. For
now I'm sticking with 2007.06 since 2006.12-SP2 had a non-DMSA bug that
caused even worse problems for me.
Compute farms:
DMSA supports compute farms. We used LSF (but others are also supported)
and it was easy to get set up. We wrote a wrapper script to tell PT about
our LSF environment and then called Chris' script to load up all the saved
sessions. Using saved sessions makes it easy to get started in DMSA. Just
take already existing saved sessions from your current PT environment and
load them up in DMSA (don't forget to start PT with: "pt_shell -multi").
- Stu Hecht
Clear Consulting LLC Santa Clara, CA
Index
Next->Item
|
|
