( ESNUG 580 Item 2 ) ---------------------------------------------- [02/23/18]
Subject: SCOOP -- a first look at the pay-per-minute Metrics SV simulator
NEW SCHOOL RTL SIMULATORS
7.) Cadence Xcelium (Rocketick RocketSim) is parallelized System Verilog
across 100's of Intel CPUs. Benched 23X faster vs. VCS, Incisive,
Questa. Does gate and RTL sims. Compiles 1 B gates in 2 hours.
Does 4-state-logic for X. Full System Verilog and accelerates SVAs.
Xcelium got #3 User's Best of DAC'16 last year. (DAC'16 #3)
"This is the industry's first *working* 3rd gen simulator, baby!"
Customers Intel & Nvidia. HPE talking about migrating to Xcelium.
ARM sees 5X speed-up for RTL/gate; STmicro at 8X for DFT simulation.
(booth 107) Ask for Uri Tal. Freebie: Denali party tix
WHO TO BELIEVE?: Aart talks about his Synopsys Cheetah VCS as if it's
his own Rocketick-like simulator -- but my CDNS sources say: "Ask Aart
to name specific Cheetah VCS users. He can't. He's just messing with
our Xcelium sales with roadmap talk." Who should I believe???!!
(booth 147) Ask for Manoj Gandhi. Freebie: pens
- from Cooley's Cheesy Must See List for DAC'17
From: [ The Black Panther ]
Hi, John,
Please keep me anonymous.
I liked your coverage of Cadence Rocketick and Synopsys Cheetah at DAC'18.
Although this isn't a benchmark per se, I wanted to share with your DeepChip
readers a first look at something different at Metrics Technologies.
I have been using it for over a year evaluating their cloud simulator and
its potential use for my company.
What's different about Metrics is they sell a System Verilog simulator that
you rent by the minute on Google Cloud. Current pricing is $.04 per minute
per simulation job.
Metrics lets you:
- Scale the number of SV simulations that can be run in parallel
with the "peak" simulation needs that often arise during projects.
- Save money as fees are based on simulation time used, not yearly
license fees that traditional EDA vendors charge.
Their product is "Metrics Cloud Simulator & Verification Manager" which is
a mouthfull -- so I nicknamed it "CloudSim" for short. As the long name
implies, it has an added verification management tool to manage regression
runs, triage failures, merge results and debug waveforms. (Everything is
in the cloud.)
I focused on the cloud simulator and not the verification manager features,
although I have used their preliminary waveform debugger to make sure the
SV simulations were actually being run.
My primary focus so far has been testing Metrics' System Verilog features
and its gate-level simulation support. This is because as the early use
cases where I can see us using Metrics would be offloading our:
- Long running bug hunting regressions
- Time-consuming gate-level timing signoff regressions.
When we need to run multiple timing corners on large test suites, being able
to scale to thousands of simulations in parallel -- for no *significant*
additional cost and to pull in tapeout dates by days is very enticing.
---- ---- ---- ---- ---- ---- ----
HOW METRICS CLOUD SIMULATOR MEASURES UP TO THE BIG 3 SO FAR
I have way too many years experience using the VCS/IES/Questa simulators
from the Big 3. This is a first look comparing the Metrics simulator to
all of them.
NOTE: We have not yet done a full performance or feature comparison, as our
initial evaluation was to see if the Metrics CloudSim simulator was real at
all, and if so, how we could potentially use it going forward. We have
been using block level UVM environments to evaluate the Metrics simulator
and are not running production projects with the simulator today.
My comments below are with respect to the *most recent 2018.01 version* of
the CouldSim simulator, which has evolved with time. (With an earlier beta
rev we had seen some constraint solver issues, and small tweaks required to
get our code to run on CloudSim.)
1. Metrics CloudSim runtime
- Right now the Metrics simulator is certainly slower than the Big 3
simulators. A typical regression run with CloudSim gives a total
simulation time is 30%-40% slower, with worst case individual runs
being up to 2x slower -- but this can be easily offset by upping
the number of Cloudsim simulations you run in parallel. In theory
your regression time could be as short as your longest individual
test run.
2. Metrics CloudSim System Verilog IEEE 1800-2012 compliance
- We have not yet done a specific feature check, but have 3 block
level environments running
- We're still evaluating SDF timing annotation support. We have
been working with Metrics to resolve some issues, and Metrics
claims they will have fixes in a near term release.
- We have not tried their Direct Programming Interface (DPI) yet.
3. Metrics CloudSim code reusability with other Big 3 simulators
- We have no current issues. The initial versions of the simulator
we used over a year ago had small tweaks required, but we are
currently able to re-use our block level environments as is.
CloudSim seems 100% compatitable with VCS/IES/Questa.
4. Metrics CloudSim constraint-solving support
- No known issues.
The constraint solver is improving, as Metrics works on their
solver optimization algorithms, and their cost model still makes
the Metrics simulator attractive.
5. Metrics CloudSim SVA checking
- No issues observed, but we've only tried simple assertions.
6. Metrics CloudSim functional coverage
- Works fine, we've observed no issues. This is specifically for
individual tests. Have not tried merging or analysis with the
Metrics analysis platform.
7. All standard test bench and RTL constructs
- No issues observed, other than the SDF file support.
8. Metrics CloudSim UVM support
- Our basic UVM testbenches are able to run on the Metrics simulator.
I suspect there are still some lingering issues to be found.
---- ---- ---- ---- ---- ---- ----
HIGHLIGHTS ON THE CLOUD APPROACH
1. Cloud Workflow
We used these two approaches our evaluation:
- Git Repo: For teams are already using GIT for hardware development
it is a straightforward transition, with probably a day or two of
initial setup when first moving to the Metrics Cloud.
- Direct upload via FTP: When I first started looking at Metrics
this was the only option, and once the SSH was setup it worked
fine, I was able to upload data, and then it was just like working
on my own company network.
2. Cloud Capacity
- We need to test how high the 'unlimited' cloud capacity can
actually go in the real world. We have asked for 1000 servers to
be brought up and were told that is no issue. Next step 10,000?
4. Cloud Interface Options
- I have personally have only launched simulation jobs from the
terminal, others in my company have used the web interface, and
the Metrics team supporting us has run simulations for us using
the web interface.
5. Cloud Security
- A few years ago our corporate policy on SaaS in the cloud was
just "no". But recognizing the benefits of this model has led
our Corporate Security department to develop a process for
vetting cloud-based SaaS providers to ensure that our security
needs are met. We are currently working through this process
with Metrics.
- Once Corp Security is happy, teams are free to use the SaaS
services.
- Metrics runs on Google Cloud Platform (GCP). When you start
with Metrics you get a dedicated cluster of machines. Our
corporate security group is still going through the vetting
process here also.
- Metrics does have encryption, which we have tried and it works
in at least one of our environments.
---- ---- ---- ---- ---- ---- ----
METRICS' PAY-PER-USE MODEL
The pricing model for Metrics is very simple. Once you are customer and
Metrics has setup your partition on the cloud, your cost is based purely on
the time used to run simulation. If one month you run no simulations you
don't pay a cent, if the next month you run 100,000 minutes of simulations
at 4 cents per minutes, you pay $4,000 for that time.
With Metrics' pay-per-use model, you only pay for simulations that are run,
and the time it took to run them. Server costs and maintenance are covered
in that pay-per-minute fee.
This cost is extremely competitive when compared to the traditional Big 3
simulators. Even if you were to run a single simulation continuously for
365 days, the cost using Metrics may be less than what you are currently
paying, when you factor in server, maintenance and license costs to do the
same on your internal network.
For small teams on tight budgets this is powerful. In the early stages of a
project, you may only need a few simulation licenses for your team. But as
you move into heavy verification you may want 100's of licenses for bug
hunting and coverage closure.
- With current EDA vendors, you must have these 100's of licenses
ready to go to support your "peak" usage needs -- which means
you are wasting money as they sit not being utilized.
- With Metrics, you pay for what you use, and you can budget your
Verilog simulation costs accordingly.
While Metrics' simulator is not quite on par with the Big 3, I fully expect
it will be within the next year, as more companies try it out. That being
said, its current functionality is enough to make it worth time to explore.
Metrics biggest advantage today is that it is providing a cost-effective
alternative to handling simulation/regression "peaks".
I see us taking advantage of Metrics in 2 ways in the near term:
- Lets our development teams increase their simulation capacity at a
predictable cost and without impacting other teams in our company.
- Lets us pull in tight tape-out schedules by enabling massively
parallel regression runs during "peak" utilization periods.
I'm excited about the potential of SaaS for EDA, and specifically that there
is a new option to compete with the Big 3.
- For smaller companies and startups, Metrics can level the playing
field significantly. One internal analysis showed how a startup
company could easily deploy more simulation capacity than our
entire company for far less than what we currently spend.
- The "Big 3" have most companies over a barrel when it comes to
simulation. We need their VCS/IES/Questa simulators, and they
know it. Having a viable alternative that is easy to incorporate
with current flows, is going to be good for us.
Beyond Verilog simulation, I see potential for Metrics and the SaaS model
to open the doors for innovation and lowering costs in EDA. For example,
formal tools using SaaS pricing are something I want to see ASAP, as their
usage is very "peaky".
We live in an "adapt or die" business.
- [ The Black Panther ]
---- ---- ---- ---- ---- ---- ----
Related Articles:
Nvidia gets 23X faster, 80% less memory with VCS plus Rocketick
Anirudh defends JasperGold's rep and the CDNS-RocketSim purchase
IC Manage PeerCache and CDNS Rocketick get #3 as Best of 2016
Join
Index
Next->Item
|
|
