Unit testing and integration testing for system models of large or medium complexity in SIMULINK
PROBLEM
I have a fairly large model of a hydropneumatic system, consisting of about 20-25 different subsystems. Each of the subsystems then consists of digital logic, edge delay blocks, and a gateway to external output ports (real world outputs). Also, some of the small building blocks are old C code imported as S-functions to maximize the cost-benefit ratio. Each of the subtype models has been designed in SIMULINK using a basic block, i.e. Additional commercial units were not used (eg aerospace, simscape, simMechanics, etc.).
The main problem is that I don't have enough knowledge about testing models in SIMULINK. I know there is a custom unit testing framework that Mathworks provides for testing and validation. The trouble is, I'm not entirely sure how this will relate to my domain. Also, my subsystems are quite complex themselves and unit testing each one is a nightmare. BUT, if checking the black box with inputs and outputs is the only way, so be it, and I'm glad to accept it.
While my question is about testing large-scale and complex systems in SIMULINK, my goal is to get suggestions from experienced SIMULINK users who have done this in the past and will do so a lot in the future. I used to test unit tests on SIMULINK, but all those tests that weren't close to my correct design.
Any help would be appreciated!
UPDATE AFTER @PHILGODDARD COMMENT
I forgot to mention above that I am familiar with HIL and PIL. However, they are only valid when you have a real-time target platform available for loop testing. What if someone wants to do soft testing in real time?
I am waiting to go through a massive Mathworks system testing webinar in a day or two. Hopefully I can get some better answers / suggestions through this?
UPDATE AFTER @ AM304 COMMENTS
To clarify the context a little further, we simulate all interacting systems in software without involving any physical devices, that is, modeling and simulating all systems in software, with the outputs being displayed on the operator / instructor terminals .... for example, we have electrical, air conditioning and hydropneumatic systems working together, but we modeled them in software. therefore, when we simulate it, all the necessary signals are generated from the behavior of the software models, no actual hardware / physical device is involved in delivering these behavioral outputs.
source to share
I would look at Simulink Design Verifier and Simulink Verification & Validation . Specifically, they offer features like automatic test generation and coverage analysis, which I think are applicable to your problem.
As a side note ...
As far as the matlab unit testing framework you mentioned in your post, it is basically MATLAB's xUnit implementation . As such, it will certainly come in handy if you want to use xUnit templates to test your MATLAB code.
The frame can be used to write tests for a Simulink model, but keep in mind that you must be able to write tests in MATLAB code. So this means using a Simulink CLI (for example, commands like set_param
and sim
) to set up and use your model, and then use framework validation techniques (for example verifyEqual
) to compare actual and expected results.
source to share
C-code as well as Simulink models can be tested using TPT. There is no difference in test design as well as test score, but in testing environment. To test C code, you can decide whether the C code should be tested in the so-called SiL mode in Simulink, where the C code is embedded in Simulink as a so-called S-function. S-function generation can be done automatically using m-scripts that depend on the code generator.
For Simulink Coder, the following MATLAB commands can be used to force SiL:
set_param(<testFrameName>,'RTWSystemTargetFile','rtwsfcn.tlc');
set_param(<testFrameName>,'RTWTemplateMakefile','rtwsfcn_default_tmf');
rtwbuild(<subsystem to be tested>);
For the TargetLink script, the TargetLink "tl_built_host" and "tl_set_sim_mode" commands are used:
tl_build_host('Model', <testFrameName>, 'TlSubsystems', <subsystem to be tested>);
tl_set_sim_mode('Model', <testFrameName>, 'TlSubsystems', <subsystem to be tested>, 'SimMode', 'TL_CODE_HOST');
Alternatively in TPT, C code can be tested in two other ways, which do not require Simulink, but some manual programming and compilation.
The first alternative is to use the so called EXE platform where the test harness is built in C code and compiled by the user with his own compiler. Test harness generation is supported in the EXE platform configuration dialog in TPT.
The second alternative is the so-called FUSION platform, which is a collaborative simulation environment. FUSION is an open architecture where the user can adapt their own system under test C code using a well-defined and documented API. The C code together with the API forms a so-called node, which can be modeled on the FUISON platform as a separate node or together with other nodes. In the case of automatic code generation, however, most users use MiL and SiL testing in Simulink, because this can be done fully automatic, including backward regression validation between MiL and SiL.
Why should you test C code instead of models? The reason is that mostly functional development is done in floating point notation, and implementation in the target ECU must be done in fixed point. The process of scaling and calculating a fixed point, at least for TargetLink, performed during code generation. Thus, floating point (MiL) results must be compared to fixed point (SiL) implementation as scaling leads to a lot of errors due to experience.
If 100% claim or coverage condition is TASMO function in TPT. TASMO attempts to automatically generate test cases to maximize coverage of the Simulink or TargetLink model. The algorithm is optimization based and search based. Please note that automatic test case generation should not replace functional test cases. Coverage targets can be validated with code coverage tools such as CTC ++, V&V toolbox, or TargetLink, which provides its own measure of coverage. By considering the coverage results, the user can independently decide how to stimulate blind spots and can identify the dead code.
I'm one of the TPT developers. For more information on TPT, you can visit our website.
source to share
MathWorks changed its V&V proposal in R2017b (September 2017). I recommend taking a look at the following links to see how they can help with device layer testing and integration in Simulink / Stateflow.
- Simulink Test - Provides a framework for testing your Simulink / Stateflow design
- Simulink Coverage - Creates coverage metrics to determine how much of your design has been tested.
- Simulink Design Verifier - Automatically generates tests to meet functional or coverage criteria.
The MathWorks website has an overview page that provides more information on the overall proposal for validation, validation, and testing .
source to share