Magazine Reviews

Just wondering if there is any plans or would be interested in listing magazine review scores for each game e.g. Zzap!, Commodore Format, C&VG etc

Might be interesting if implemented right. At the moment you can only write the scores as a comment type text which I think is not an optimal way to present it. Maybe a special article type for magazine review scores (or website review scores) could be created? I would support that idea.

Edit:
Off-topic. Try to remove the borders from the C64 screenshots if you can. I know its a bit of extra work, but it looks better in their normal 320x200 resolution without borders.

If magazine review scores are included, website ones could be as well (I'd really like to have Metascore because it's actually been used in publishing contracts). Both of which may require some form of agreement on their use (might give additional click-through revenue for covering UVL's hosting costs).

Borders are now gone. :)

I like the idea of including magazine scores. I would also like to see a system for UVL integration. But honestly I think other information is a priority and wouldnt want addind such a system to take to much resources.

I totally agree with Zerothis.

I like the idea too. I was already thinking to display metascores too.

I tried to find any evidence of some sort of agreement about using this and other scores, but nothing came out.
I think this should not be a problem if correctly credited.

About the integration, Teran is right about a custom article type, along with a snappy UI to input the data. This shouldn't take much time.

Alternatively tags and their notes. Whichever works better.

I'm working on it. I was almost there, but I need to check some bits.
I'm writing this just to verify that the classic article editing works fine :-)

Because of this kind of stuff too...
www.gamasutra.com/view/news/169475/Creative_Assemblys_fairly_brutal_approach_to_achieving_high_Metacritic_scores.php
www.gamasutra.com/view/feature/169354/how_creative_assemblys_process_.php

The first draft is ready.

On the game info page, on the right-side editors area, there is the "External reviews" section.
The "New external review" will bring a page, similar to the article editing page.

The main article is not needed, and there are the new fields:
- Source (C=64 magazines list so far)
- Issue (if applicable)
- Date ( classic ISO format used in UVL: YYYY or YYYYMM or YYYYMMDD )
- Score (a percentage, so a number ranging from 0 to 100)

- The "source" has to be a drop down list, to have consistent data and prevent typos and similar. So it has to be expanded "on request".
- I think to make it smart, so if you are going to add a score of a C=64 game, only related sourced will be shown (i.e. no Your Sinclair)
- The review link is used to point to an external site with the review. We are not going to show the original review text in UVL. Just the score.
- The main body text looks useless, but could become useful in other cases.
- Non magazines (review sites) can be included, always providing the link to the original page.
- The rating from 0 to 100 means the other rating systems (i.e. 5 stars) should be converted. Handling different methods at the same time looks cumbersome.

As usual, ideas are welcome.

Be prepared for many requests at the beginning then. I hoped we had some kind of info page about the review source. With the possibility to edit it. Maybe shortly describing the life-cycle of a review source (e.g. old magazines like Zzap). That it would look similar to platform info pages with the possibilty for us editors to at least add an article about it.



Good idea, but also lots of work at the beginning because of multi-platform magazines.


I see no link. Only links I see is "edit" and "New external review". I actually find it rather useless right now. It only appears in the editor area. This has to be visible for every user that is not logged in (*). The text is everything in lower case and written together. Something like commodoreuser is not well readable. Or was I just blind and didn't find the spot where reviews should appear in the game info page?

(*) The same should apply to the rating system btw, which is also only visible by logged-in editors (but I consider the ratings sytem beta and not much used anyway).


Would rather see the real rating of a magazine like 9/10. Or the Famitsu x/40 score. Then I would like to see some kind of "meta"-score at the end. And here its useful to automatically calculate the 9/10 rating as 90% or something.

As I wrote above, this is the "first draft" :-)
The first step was to setup the data and the editing page needed to make it work.
I made it visible to have feedback and find the right direction before going too far.

The "output" right now is very basic and visible only in the editor area. The source is just a keyword (i.e. commodoreuser) and the direct link is just stored now.

At first I was thinking about using the groups to store magazines data, but looks forced as no games would be tagged.
The sources management is in the works, mainly to store the min-max values needed to the conversion that needs to be done.
Are there review systems that are using non numerical scores?

SeanCapoue will provide the data for some popular magazines. When the data structure is finalized, I will import that data.

-edit-
Please don't start adding many, the data will be erased at the next update.

Could be used for notes about the review, like who wrote it (tho the editable author field of normal articles is more useful for this) or anything unusual about them.

Magazines manager!
/reviews/sources

Still not connected with the reviews, but another step toward the right direction.

Feel free to add new items, and share your thoughts.

-edit-
I modified this new section URL from magazines to reviews

Wanted to bring the following site to everyones attention, because its a superb resource for computer (gaming) magazines. Many full scans of mostly older magazines, which can be read conveniently online.
http://archive.org/details/computermagazines

The UVL magazine review score section could shape into something very nice I think. Has lots of potential, could also lead to some interesting statistics.

Yes, it is a great resource!
While testing I have added a bunch of magazines. I had some weird refresh issues, but I just realized it is Chrome fault ...
Please let me know if there is something wrong.

Cheers AndreaD for implementing it. Just tried it and works fine. I have emailed a couple of points but nothing major just very minor details which I'm sure your aware of.

The website teran01 reccomends is what I use and is excellent

archive.org/details/computermagazines

Cheers guys

Sean

The date field seems to accept "malformed" dates, as seen here: /users/logs/game/19804

How do we deal with reviewers that have changed the way they present their ratings (e.g. changed 1 to 5 rating to 1 to 10 and later into percentage)? Or with reviewers that don't provide aggregate rating for games (like, they give separate numbers for each category they rate games with but no general rating derived from them)?

For the latter it seems the best idea is to average them out and expand the outcome to 100% equivalent (e.g. with number of 1 to 5 ratings, 3.2 average would be transformed with ×2×10 into 64%). And write the specifics into the article body part to clarify it.

It should be noted that some reviewers have been so impressed with a certain game that they have done things like Using 5.5 stars in a 5 star system or 110%. There are also other occurrences of a game magazine modifying their established systems.

Great! I didn't see this, I'm just trying it.

Do reviews in other languages can be accepted? Some machines were not or barely distributed in English speaking countries so it could be difficult to find English reviews. If positive, maybe we could choose the language of the review in the "new review" page, unless it's connected to the magazine / reviewer, so we may choose this language on the "add magazine" page..

Do "issue" means "n° of the magazine"? I'm not sure.

Usually it is.

I added two magazines and some ratings, everything looks fine for now.

Could "review link" links to anything? I linked it to a PDF file of the archive.org website, is it fine?

I assume it should be. We don't post the review text and for the "score" the language is not relevant.


Yes, although I there is a good bunch of magazines that do count their issues in numbers.

Edit:
I'm still waiting for how it will look llike when the reviews/scores are finally implemented in the UVL info page. Until then I don't have a real opinion about it.

I hope it will be something like the first three columns like they are in HOL with having mag name, issue/date and score. Keeping it simple but somehow nicely integrated in the UVL game info screen.
http://hol.abime.net/1556/review

Probably another tab for the random information below the main information chunk.

I've been a "bit" busy these days. I'll work on it this weekend and make it usable.
I'm a bit sad about the departure of SeanCapoue that started this thread, btw.
Feel free to add more idea / questions, as usual.

Was it not usable already?

Not sure if this still applies. If I were you I wouldn't put too much effort into entering reviews yet until the testing phase is over.

I had some time available these days, so I entered many of them, as I like seeing how "old" games were perceived back then. I noticed the "please don't add many for now", but I thought it became obsolete already.

Loosing data was still possible, but I think it is quite solid now, so your work should be safe.

Great then !

It may be a bit too early to talk about this now, but I think that scores should be regulated according to how the magazine rates the game. I noticed that old magazine, especially if the rate is out of 100, are very generous. I remember that when I was a kid reading magazines, a "not so good" game had a score of 79% or less (a very good game had at least 90%), or so I considered that. Whereas now, websites I regularly consult rate 3 out of 5 (60%) or less a "not so good" game.

For example, if a magazine has a mean rate of 70% whereas every rate of every magazine have a mean of 50%, then we could consider that this magazine is too generous, and lower the average rate of all its reviews (maybe decreasing its socre with (70-50)/100 = 20%). Both score should be visible, though.

This would be available only in the future, when average score of each game will be available to everyone.

This is what Metacritic does, they see rating practices of every source they include and adjust the "average" accordingly. 1 to 5 and 1 to 10 ratings tend to be reliable, while % (or 1 to 100) ratings tend to gravitate to putting average at around 70.

"Review" filter should be added to the "What's up" page.

Adjusting review scores? Metacritic really does it? That tends to lead me into the "we should not mess with calculating average scores then"-group.

Giving too generous average final scores are not really something that bothered me with reading certain magazines. There are two other points that I had always trouble with (but which are not relevant to UVL it seems)

I give two examples:

1.) Something I see very often in old British magazines (like Zzap or Crash). Lets take two similar games of the same genre. Both have the same graphical quality and sound quality. One game plays bad, the other one plays great. Now the bad game gets a 40% and the good game a 90% as end score. Graphics and sound would be the same score you would think.... but: wrong. In nearly all cases the graphics and sound score of the bad game would be around 50% (not much away from end score) and the graphics and sound of the good game would be around 80% (again not much away from end score). This is so blantantly obvious sometimes. The single scores are extremely homogenized. They thought: "Oh a good game. Now we must give every single part of the game a good score, even if its only average in that aspect". The result is that very good "Puzzle" games would get sometimes higher scores for graphics than artistically well done but bad playable adventure or action games.

2.) I hate it when the end score is strictly mathematically calculated by single scores. My main complaint goes to the best-selling gaming magazine in Germany , the "Computerbild-Spiele". The have a very very detailed score box, with many scores. Each score would influence the end score be a certain percentage. Lets say graphics would influence by 20%, sound by 20%, even laughable scores like "can the package be recycled" would influence for 1-2% or how well the installation routine would work (5%).
Now there is the typical occasion when you have a new decent budget game which got a higher mark than Final Fantasy 10 due to elements that have nothing to do with how fun a game actually is (price, package, some translations errors in FF10, while the very few text in the budget game was flawless). This is plain stupid to calculate a final score. Not sure if they do this still, but some time ago (when I bought some issues only for the full versions of games on their magazine cover CD-ROM) it was the case.

So far you have entered 1-100 scores.

The link to HOL highlighted the fact that there are many ways to keep scores, we have the following:
515/1000
5/10
3.5/5
79%

Please note that the third example is a decimal number! (just like our half stars)

So I'm going to have the editor to enter the original score, convert it to a 1-100 range (for easy sorting and eventually computing averages), and store both.
This means that the user interface will act differently according to the selected magazine.

Do you know other ways used to handle scores?
For example with a A to F ( with partials like C+ ) would be at least "annoying" to handle, and maybe a simple "thumbs up" "thumbs down" has been used too (how to convert it? up=100 down=1 ?)
So instead of just storing the max score for each magazine, I should introduce a "score type" to handle the different behavior.

The only bit that doesn't cover is when reviewers don't provide an overall score.

Some magazines use scores /20. But for now, I don't know any which uses "letters" scores.

The reviews should be ready.

It accepts different types of score, that are correctly handled internally.
When selecting the source, the score input reminds you the type of score that needs to be entered.
There is no proper checking, so you could type 200 for scores ranging 1-100, but at least really wrong inputs are simply not accepted.

Next, I will work on showing the reviews to anyone. For now it will be a tab, as expected.
Consider that the info page has not been updated to the new layout yet, considering it shows the most important content of all the site, I wish to make it in the best way: readable, not cluttered but still informative, while maintaining the responsive layout. I have already done and then scrapped a few versions ...

I remind you that there are two related pages:
the sources list : /reviews/sources/
and the latest 20 reviews /reviews/list/
Filtering the reviews by source and more will came next.

The reviews(s) tab is now active.
I added some mixed reviews to /game-17216 to check various issues:

- Different ways to handle the issues
- Year-only review dates
- Different scoring systems.

Looks like everything works fine :-)

(I have added a new tab too, "contributors")

Might want to label them as external reviews unless user reviews will be thrown in the same list. Or otherwise improve the distinction of the review article type ("personal review") and publication/critic reviews.

BTW the example game Vampires Empire is a good example to put the Media tab at least behind the tag tab. It looks strange to see the big video window immediately after opening the info sheet. I think I requested that before.

Otherwise: Well done. I will definitely contribute some external reviews. Mostlikely I won't crosslink to the external source. I fear that when the link goes down it becomes a mess of broken links. (Will go through the video links once again, because I see some broken links there already.. and at least they are easily spotted - reviews will be impossible).

Scores are not visible anymore on the right tab? For example, Landstalker:

External reviews
Mean Machines Sega issue:12 date:199310 (%)

Updates:
- Labelled to external reviews
- Moved the media tab
- Scores in the right side are back (in %)

I agree with the broken links issue. But no idea how to fix it, it is the typical internet "achilles heel" ( and no, I'm not going to store huge pdf files here..)

When generating page cache, the code could check if the URL leads to valid (not a 404, 503, or other non-200 HTTP response code) non-redirecting (no HTTP or HTML redirect) page like every 3-6 months since the link was added. If that changes, it could be marked and reported as a link needing manual checking and updating (e.g. archive.org can sometimes be used to link to cached copy of one that's no longer valid).

Yes, it does. The final score is adjusted depending on the "generosity" or "severity" of each reviewer. Moreover, it also weights each score with a coefficient applied to each reviewer, depending on how serious the reviewer is... but they refuse to explain how they judge this "seriousness".

I thought it was just adjusting the usual range they score things, like if a review source with supposedly 1 to 100 rating only uses 60 to 95 scores, they'd stretch that so 60 corresponds with 1 and 95 with 100 and whatever they commonly claim as "average" adjusted to 50. Essentially stretching the original score to their 1 to 100 thing with average dragged not exactly 1:1 with the endpoints.

While having encountered no problem with entering Mean Machines Sega reviews, I now encounter some with Mega Force.

The magazine changed its notation system. In the beginning, there was no overall score, only 1 to 5 scores given to 4 criteria, then, they changed to % scores. Sanguine already mentioned both issues, that both applies to Mega Force.

Another issue is that the magazine sometimes applies several scores to the same game. For example:
95% if you don't already have the previous episode, 60% if you have it.

Or:
If you have the lightgun: 85% for one player, 90% for two players. If you play with a gamepad: 40% for one player, 49% for two players.

Or:
If the player is young: 88%, otherwise: 68%.

For now, I only added % scores and did not add any for games that have several scores.

(Personal note: Lethal Enforcers, MF n°21 / Virtual Pinball, TMNT, Wiz 'n Liz, Madden NFL 94, MF n°22 / Dino Dini's Soccer, Yogi Bear, VR Deluxe, Star Wars Arcade, MF n°34 / Bloodshot, ATP Tour, MF n°36 / Wayne Gretzky, MF n°39 / D, MF n°47 / WWF Wrestlemania NES, JP n°4 / Flintstones NES, JP n°6 / Berlin Wall MD, JP n°7).

Could be acceptable to add two magazines, before and after the change?

Seems acceptable to me, but I don't really bother right now as we don't know how to handle magazines that don't use "overall score" yet, and I'm not sure we should add this type of review unless we detail every score of every criteria, which would be a big mess.

The more concern I have is about this "multiple scores" to a single game in a single review. I thought about two solutions, but I don't like them much:
* Use the average score
* Add multiple reviews (with the condition of the score stated in the comment).

I prefer the second solution, though.

I tend to think the highest score should be the one used. Because it doesn't make a game worse when it is build around a certain niche setup (lightgun, multiplayer) to reach its full potential.

As for multiple scores -> maybe thats where the "text field" of external reviews (at the moment not really usable) could work? Or could something like the note for tags be implemented here to describe these special cases?

In other words, playing a game with less people or with setup it was not primarily meant for is extremely likely to produce less good results.

You're right, I totally agree with that, I prefer this solution. Besides, that's the solution I chose for the first review that used this multiple-conditionnal-score system (the condition was: if you do/don't already possess the previous instalment in the series - I considered then that a game should not be rated according to the games the user already possess).

This is a bit iffy. Is it a good game on its own, vs is it a good game as a part of the series. People who care about the series likely also care about the latter condition more than the former. Usually result of the game doing nothing new compared to its predecessor and isn't driven primarily by narrative.

That was exactly the case. Not "nothing new", but "not enough things new".

EDIT : I noticed that when entering several reviews by the same magazine to a single game (that is possible, examples: reviewed once in import version, once in PAL version ; reviewed once a prototype, once the commercial version ; reviewed once by one member, once by another member ; etc...), in the "External reviews" tab, only one review appears. The number of reviews shown in brackets correctly counts the reviews, however.

Think this would be covered by previews with press version, pre-release demo, actual demo, or other such thing.

Previews for demos, yes, but I'm pretty sure some games reviewed and given a score by a magazine were only prototypes. Such as games that were never published in the end.

I think it was common to review "demos" just to have some big name on the cover before the others...

Thank you dandyboh, I didn't notice this issue about multiple reviews in the same mag.

-edit-
Fixed.

I think that are generally copies with the complete game send from the publisher to certain magazines earlier than others to get an exclusive title story (during the time where the game itself is "finished", but not yet published/pressed on CD-ROM etc).

I only remember one time in a German magazine where they reviewed Lemmings 2 and I am absolutely sure it was based on the demo. I owned the demo which consisted of only a handful levels that were not in the final game and they had only screenshots from the demo levels. And the text never hinted on facts that you could only know by playing the full game. Should I say more than that this was the very last time I bought that particular magazine? This is a no go for me.

Well, I'm sorry but... no. I have an example of a magazine, which did reviews for Sega machines games only, but began to review PS1 and N64 games at the end of 1996, when the PlayStation had already won the match against the Saturn in Europe, and the magazine did not sold well. But, for these games, they did not score %, as they did for Sega games, but out of 5. So the same issue could have different scores, % for Sega games and out of 5 for other games... a bit complicated, I admit it.

Just do Magazine (PS1) and Magazine (Genesis) or something in that case, maybe? Suboptimal, but it's an option.

I'm wondering: Could it be better then to not have the review score system fixed for a certain magazine, but per single review? It would clearly mean that you have to do a little extra work per review (choosing the type of scoring system for each review), but at least you can cover each and every review then.

It would be a shame to not have have to enter the data just because there are those (almost rare) special cases.
I would add the option to override the default, but I have to review the code and the data set, not something I can do this week. ;-)

OK, thanks.


I think it would be useful for editors (and for other users?) to have a list of issues per magazine (grouped by issue + date). Something like:
Issue / Date / Number of reviews / (maybe average score, min score, max score)

This would be useful to:
* Find mistakes on issue number and/or date (for example, a magazine that has 15 reviews for issue #20, date 199501 and 1 review for issue #20, date 199401 => there probably is a mistake on the date or the issue number for this review). Maybe clicking on "15" or "1" would lead to a result page with every games reviewed in this issue.
* Track issues per magazine that was not browsed yet by an editor.

For end-users grouping review sources per country/region (and language?) might be useful. Like German people might be more interested in German reviews, so have them clearly there (and maybe later have user configuration for which regions to prefer and show those at top?).

I've also heard enough many times things along the lines of "well received in Germany/Japan while not so much in the rest of the world" and I think there were some games that were surprisingly poorly received in their country of origin but much better elsewhere, so giving that visibility somehow would be neat as well.

A lot of good ideas here.

- In the sources page the issues# and reviews# column are added.
- In the same page, you can click on the issues value, to view the list of issues for that magazine, with some stat (based on the % score)

Lot more could be done, this is what I could do today.
(I suggest to start a new "reviews ideas" thread, btw)