As some of you know, we've got a set of claims from visitors of this web site. So we decided to compose a list of major of them and describe the decisions we're going to implement that finally must eliminate all the mentioned the issues. So let's go:
1. Tests are wrong - they're meaningless, unrealistic and so on.
We will:
- Write articles & FAQ explaining what we've been testing much deeper, as well as the importance and applicability of results.
- Implement at least one of standard TPC tests (TPC-C / TPC-E).
- Add CUD tests with explicitly disabled batches; change "CUD average" derivative formula.
2. Tests are intentionally written to score DataObjects.Net better; other ORMs are misused on these tests
Ok, let's be fully honest: I think this claim is simply lie. There were no any evidences approving this, and I believe they won't appear. It was initially shown there is nothing to hide for us. You can download the source code of this test suite, DataObjects.NET (it's shipped with full source code as well) and test everything by your own. Run it under debugger\profiler, see what it happens there, find out if something special is done for any of these tests. I'm ready to bet you'll find nothing.
Moreover, as we wrote, this test isn't "designed" for DataObjects.Net architecture - on contrary, it's one of the worst tests for it. E.g. we knew we must loose in materialization test to ADO.NET Entity Framework. I'll explain what sort of framework might be high-scored on our test suite shortly in FAQ.
Anyway, DataObjects.Net is already removed from test results at all - at least to stop any argues around this.
We will:
- Provide links to test files in each participant's page. This will allow all of our visitors to validate the test code.
- Publish full source code on Google Code (btw, already done) and invite test co-authors and experts there.
3. This site is nothing more than DataObjects.NET marketing resource
There is no more DataObjects.Net here. Hopefully, this fixes the main issue.
We want it to be a marketing resource for any good ORM tool on .NET.
- All you need is higher score.
- As I'll shortly explain, no one can get it with just some simple tweaks. The company must be an expert in .NET development & ORM to be well scored here.
4. This site is private and closed resource. Thus there is no room for open discussions here.
Already done:
- Comments are enabled for all the pages.
- Discussion forum is fully open now (earlier it was necessary to register here to access it).
- Test suite source code is shared at Google Code.
- Online source code browser is added to this website.
5. The name of this web site implies agression. Thus it looks like #2 and #3 is true.
Initially we took orm-comparison.net and orm-comparison.org as well. Actually I wanted to give this site a bit unusual name associated with orm comparison to make it easy to remember. Anyway, we'll change its name after some marketing research and visitors polling.
Hopefully, this will help to make ORMBattle.NET better.
P.S. If I forgot about something important, please leave us a comment.
Kind regards,
Alex Yakunin.
