[oi-dev] COMPONENT_TEST_TRANSFORMS for PARI

[Aurélien Larcher]

On Sat, Oct 15, 2022 at 7:34 PM stes at PANDORA.BE <stes at telenet.be> wrote:

>
> http://docs.openindiana.org/dev/userland/
>
> requires an update to first define "test" as a TARGET
>
> because test is not listed in the table after
>
> "To build a component you simply cd into the directory of the software,
> and type "gmake TARGET" (here we use gmake to call GNU make), where TARGET
> can be one of:"
>
> some targets are listed but "test" is not one of them.
>
> Further down there is just one paragraph as documentation about the whole
> process:
>
>
"It's advised to put the expected test ouput in test/results-BITS.master
> (where BITS are either 32 or 64) and to ensure that the gmake test target
> generates reproducible results. You can use the COMPONENT_TEST_TRANSFORMS
> variable to set a list of sed directives to transform the test output and
> make it reproducible."
>
> This is not entirely correct as BITS can also be "all" instead of 32 or 64.
>
> Also from your reply I understand that there are 2 usages:
>
>   1) run "gmake test" to run a set of tests
>   2) run "gmake test" to run a test-and-compare if
> test/results-BITS.master already exists
>
> So basically "gmake sample-test" something like that could create an empty
> tests/results-BITS.master,
> just like "sample-manifest" is creating a sample manifest.
>

The maintainer's guide that I had started writing on the wiki did not make
it to the docs website (yet):

https://oi-wiki-test.madgwick.xyz/35913833.html

https://oi-wiki-test.madgwick.xyz/2.-Overview-of-the-Build-System_37060624.html

https://oi-wiki-test.madgwick.xyz/3.-Common-Tasks_37060626.html

The last page contains information on transforms.



>
> ----- Op 14 okt 2022 om 1:53 schreef Tim Mooney via oi-dev
> oi-dev at openindiana.org:
>
> > In regard to: [oi-dev] COMPONENT_TEST_TRANSFORMS for PARI,
> stes at PANDORA.BE...:
> >
> >> Because PARI/gp is using its own Configure do I have to define
> >> COMPONENT_TEST_TRANSFORMS to remove all compile/build output ??
> >
> > Probably.  Whenever I wanted to have comparable test results for
> > a component, I found it easiest to use a filter to delete everything
> > up to the start of the tests, and then additional filters to remove
> > anything that would be system-specific or variable within the tests
> > (e.g. timing for test runs, dates and times, system name, etc.).
> >
> >> I am not sure the documentation on oi-userland build is sufficient
> >> regarding to "gmake test".
> >
> > Especially for someone just starting out, the current documentation
> > alone is probably not enough.  You're not a beginner, but even you
> > have run into questions.
> >
> > When I was starting out, I got good help from Alexander, Michal,
> Aurelien,
> > etc.  Most of the time the help was pointing me at an existing example in
> > a different component, that did something similar to what I needed to do
> > for the component I was stuck on.
> >
> > What might be useful is for you to inventory the components that have
> > test output for comparison (under various possible names), and then check
> > the Makefile for these to find what COMPONENT_TEST_TRANSFORMS are in
> > place.  That should give you a lot of examples to start with.  I know
> that
> > for the perl components I worked on and any libraries, I tried to
> > make certain there was a comparison file for the test suite.
> >
> >> Any suggestions ?
> >
> > If I was updating a component that didn't have a test comparison file
> yet,
> > I would start by creating an empty one, via something like
> >
> >       touch test/results-all.master
> >
> > (or a BITS-specific variant or alternate output name).
> >
> > I would generally then start running the test suite, tweaking the
> > COMPONENT_TEST_TRANSFORMS to filter out more of the output, and manually
> > updating the starting comparison file.  Frequently this involved saving
> > "diff" output and adding it to the comparison file while stripping the
> > diff markers like "+" at the beginning of a line and line-number markers.
> >
> > As long as you're familiar with the "diff" format, it's pretty easy, and
> > if you have a bunch of components to update the repetition will get you
> > to the point where you're very comfortable with the process.
> >
> > Tim
> > --
> > Tim Mooney
> Tim.Mooney at ndsu.edu
> > Enterprise Computing & Infrastructure /
> > Division of Information Technology    /                701-231-1076
> (Voice)
> > North Dakota State University, Fargo, ND 58105-5164
> >
> > _______________________________________________
> > oi-dev mailing list
> > oi-dev at openindiana.org
> > https://openindiana.org/mailman/listinfo/oi-dev
>
> _______________________________________________
> oi-dev mailing list
> oi-dev at openindiana.org
> https://openindiana.org/mailman/listinfo/oi-dev
>


-- 
---
Praise the Caffeine embeddings
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openindiana.org/pipermail/oi-dev/attachments/20221016/4675863c/attachment.html>