I, like many of yourselves enjoy playing ESO - but also suffer some of the performance issues.
I see a lot of threads, in fact many today - on performance issues. So i thought i would have my 2p worth.
firstly let me state my credentials. im the head of operations within a UK multinational - i design (and currently manage) cloud systems utilizing an application set with billions of transactions per year. So as tech goes, I think i'm in a decent position to talk about applications, hardware and distributed IT systems.
As its just my opinion - you can take it or leave it - but id like to think if nothing else its at least a balanced, technical one,
A lot of the threads i see are along the lines of "throw more hardware at it" , "game is full of bugs", "memory leaks", etc. Well that may be true. but the first thing you look for in IT issue is evidence.
Right now, the evidence we have is what is generally termed as "end user, superficial" - basically people are reporting the "end result" and what they are experiencing is the effect, not the cause - and making an assumption on the cause, based on that experience.
Its quite a natural thing and you see it a lot in IT. For example "the games running slow, so throwing more processors at it would help right?" - actually maybe, maybe not.
hard information isnt really provided by the end user - and thats the most unhelpful bit for any support organisation.
there are several componants with ESO as i see it
* The end users hardware
* The connection between the end user and the servers
* The interpretation of that input /output at application level
* The underlying hardware and infrastructure
What this means in practice is that there are essentialy 4 key areas in the chain where things can go wrong.
1) Your PC / xbox/ ps4
2) The internet
3) The application/database
4) The servers
SO the first question that should be asked is - where is it going wrong. and actually thats quite a difficult thing to nail down.
Ive seen a lot of threads saying the equivalent of "ive got a BIS GFX card and a BIS processor so it cant be me" -
Actually yes it can. Theres lots of things that can be happening locally. How do you know your anti virus hasn't kicked in? how do you know you havent got disk latency issues, how do you know that Windows isnt trying to throttle your CPU? or something else is happening in your sub system - how do you know about your CPU threading architecture in conjuction with the ESO app. Chances are - you dont. youve got no evidence apart from "its crashing/slow" you are seeing the effect not the cause.
For arguments sake, lets just say youve got a decent PC or indeed an a PS4/XBox one X. I actually have the later - so im very limited what i can do with hardware . But factually i can tell you that the move from using the Xbox one X's standard HDD to an after market external SSD make around a 40% speed increase on ESO - So what this is telling me factually is there is a disk intensive part of the application, in turn meaning the faster disk you have, the faster the game will run! So this allows us to make some assumptions based on the application. If it does this for a fixed peice of hardware - its logical to assume that this will also work with PC's/PS4s! - so maybe people should try that? (of couse you could also argue that it should work fine without the need for an SSD, but optimization costs dev company's money!)
Anyway, lets now say you've got a £4k PC setup with a 3ms SSD and its still buggy.
So lets look at the internet.(2)
It doesn't matter if you have a 100mb Download/Upload (its my considered experience that most people exaggerate with their dl/ul stats ) - but most people are on 20/20 + these days which is more than sufficient. The interesting thing is not the download/upload speed that is truly important (although clearly more is better) - its the LATENCY,, ROUTE and PACKET LOSS to servers. Obviously you want this as minimal as possible - but this is somewhat out of your control. It depends on your physical location, ISP, internet routing- and thats before it even hits the ZoS servers. It can be mitigated by using VPNs, changing ISPs or indeed, moving house (but thats a bit extreme!) but its a factor which is largely outside your control and definitely outside ZoS control. However it can be a major factor in enterprise scale apps.
Of course, the size of the pipe OUT from ZOS is part of the hardware, but that can be monitored pretty easily and you'd assume they will know if its an issue - so that brings me on nicely....
Server Hardware (4) is actually the easiest to nail for ZoS - they will know pretty easily if the servers have run out of RAM or processor, disk latency or indeed processor thread bottlenecking.
Processor thread bottlenecking (95%) of the time is caused by application ,but ill go onto that. Anyway, its pretty easy to identify hardware issues at server level and the solution is indeed, throw more at it! so they would have already done this if it was an issue. Processor and Ram is relatively cheap. Youd have to assume they have monitors all over their systems (most IT companys do) - if they dont - then they truely are mickey mouse - but i cant think its the case here.
so now we're getting into they key parts of it. The application
Now i could go on about application architecture. But generally it my experience applications are only written for performance AFTER a problem is identified rather than during the dev lifecycle (which is often time constrained using sprints/agile) - in short , dev's write it quick and leave it for the testers to find or not. (sweeping generalisation i know - and some are more diligent than others)
With ESO theres three parts that i can make out.
1) the client side app
2) the server side app
3) the database/back end
Its entirely possible the client may have issues . ive seen the "its got memory leaks argument" - but unless someone has done a memory dump and deconstructed the application use, this cannot be proved/disproved. It could be still be a local issue (version of .net etc) - but i think you;d need to grab mem dumps to take that forward. Its indeed also possible the appliation might have slowness because of hardware related issues (see above re SSD) and it just needs optimizatation
server side app - youll never know! thats one for the devs. similarly database/back end - without knowing the bottlenecks you wont know if theres an issue . could be a table/index - read/writes - clustering issue - could be a lot of things. but id have expected a DBA to be all over that (in conjunction with the app guys)
i guess what im trying to say in short is taking an approach of "Zos sort it out" isnt going to work. to take any perforance issue forward they will need definate info. Whats gone wrong, where did it go wrong, what evidance do you have, do you have a memory dump, screenshot, can it be repeated? only THEN will they have a fighting chance of fixing it.
everything else is just noise and supposition until proved otherwise.
the only thing i can see they are "guilty" of is lack of communications. its my opionon they do need to get better with their comms base. The jobs that @ZOS_GinaBruno and @ZOS_JessicaFolsom and the team do is thankless - and they must get some sort of abuse! id ask people that they remember that its not their personal fault and to pay them due respect.
At the same time and speaking as an end user its my opinion that comms to the user base needs to be done in a "different way" in order to commercially satisfy what is increasingly becoming a disgruntled user base.
PS just my opinion and PS no i dont work for them!. i just have a lot of sympathy being in a senior position in an unrelated company doing a similar thing
im sure theres a lot of factors i haven talked about and happy to chat further with those who may have a different stance. if youve got this far. well done - have some anniversary cake.
I see a lot of threads, in fact many today - on performance issues. So i thought i would have my 2p worth.
firstly let me state my credentials. im the head of operations within a UK multinational - i design (and currently manage) cloud systems utilizing an application set with billions of transactions per year. So as tech goes, I think i'm in a decent position to talk about applications, hardware and distributed IT systems.
As its just my opinion - you can take it or leave it - but id like to think if nothing else its at least a balanced, technical one,
A lot of the threads i see are along the lines of "throw more hardware at it" , "game is full of bugs", "memory leaks", etc. Well that may be true. but the first thing you look for in IT issue is evidence.
Right now, the evidence we have is what is generally termed as "end user, superficial" - basically people are reporting the "end result" and what they are experiencing is the effect, not the cause - and making an assumption on the cause, based on that experience.
Its quite a natural thing and you see it a lot in IT. For example "the games running slow, so throwing more processors at it would help right?" - actually maybe, maybe not.
hard information isnt really provided by the end user - and thats the most unhelpful bit for any support organisation.
there are several componants with ESO as i see it
* The end users hardware
* The connection between the end user and the servers
* The interpretation of that input /output at application level
* The underlying hardware and infrastructure
What this means in practice is that there are essentialy 4 key areas in the chain where things can go wrong.
1) Your PC / xbox/ ps4
2) The internet
3) The application/database
4) The servers
SO the first question that should be asked is - where is it going wrong. and actually thats quite a difficult thing to nail down.
Ive seen a lot of threads saying the equivalent of "ive got a BIS GFX card and a BIS processor so it cant be me" -
Actually yes it can. Theres lots of things that can be happening locally. How do you know your anti virus hasn't kicked in? how do you know you havent got disk latency issues, how do you know that Windows isnt trying to throttle your CPU? or something else is happening in your sub system - how do you know about your CPU threading architecture in conjuction with the ESO app. Chances are - you dont. youve got no evidence apart from "its crashing/slow" you are seeing the effect not the cause.
For arguments sake, lets just say youve got a decent PC or indeed an a PS4/XBox one X. I actually have the later - so im very limited what i can do with hardware . But factually i can tell you that the move from using the Xbox one X's standard HDD to an after market external SSD make around a 40% speed increase on ESO - So what this is telling me factually is there is a disk intensive part of the application, in turn meaning the faster disk you have, the faster the game will run! So this allows us to make some assumptions based on the application. If it does this for a fixed peice of hardware - its logical to assume that this will also work with PC's/PS4s! - so maybe people should try that? (of couse you could also argue that it should work fine without the need for an SSD, but optimization costs dev company's money!)
Anyway, lets now say you've got a £4k PC setup with a 3ms SSD and its still buggy.
So lets look at the internet.(2)
It doesn't matter if you have a 100mb Download/Upload (its my considered experience that most people exaggerate with their dl/ul stats ) - but most people are on 20/20 + these days which is more than sufficient. The interesting thing is not the download/upload speed that is truly important (although clearly more is better) - its the LATENCY,, ROUTE and PACKET LOSS to servers. Obviously you want this as minimal as possible - but this is somewhat out of your control. It depends on your physical location, ISP, internet routing- and thats before it even hits the ZoS servers. It can be mitigated by using VPNs, changing ISPs or indeed, moving house (but thats a bit extreme!) but its a factor which is largely outside your control and definitely outside ZoS control. However it can be a major factor in enterprise scale apps.
Of course, the size of the pipe OUT from ZOS is part of the hardware, but that can be monitored pretty easily and you'd assume they will know if its an issue - so that brings me on nicely....
Server Hardware (4) is actually the easiest to nail for ZoS - they will know pretty easily if the servers have run out of RAM or processor, disk latency or indeed processor thread bottlenecking.
Processor thread bottlenecking (95%) of the time is caused by application ,but ill go onto that. Anyway, its pretty easy to identify hardware issues at server level and the solution is indeed, throw more at it! so they would have already done this if it was an issue. Processor and Ram is relatively cheap. Youd have to assume they have monitors all over their systems (most IT companys do) - if they dont - then they truely are mickey mouse - but i cant think its the case here.
so now we're getting into they key parts of it. The application
Now i could go on about application architecture. But generally it my experience applications are only written for performance AFTER a problem is identified rather than during the dev lifecycle (which is often time constrained using sprints/agile) - in short , dev's write it quick and leave it for the testers to find or not. (sweeping generalisation i know - and some are more diligent than others)
With ESO theres three parts that i can make out.
1) the client side app
2) the server side app
3) the database/back end
Its entirely possible the client may have issues . ive seen the "its got memory leaks argument" - but unless someone has done a memory dump and deconstructed the application use, this cannot be proved/disproved. It could be still be a local issue (version of .net etc) - but i think you;d need to grab mem dumps to take that forward. Its indeed also possible the appliation might have slowness because of hardware related issues (see above re SSD) and it just needs optimizatation
server side app - youll never know! thats one for the devs. similarly database/back end - without knowing the bottlenecks you wont know if theres an issue . could be a table/index - read/writes - clustering issue - could be a lot of things. but id have expected a DBA to be all over that (in conjunction with the app guys)
i guess what im trying to say in short is taking an approach of "Zos sort it out" isnt going to work. to take any perforance issue forward they will need definate info. Whats gone wrong, where did it go wrong, what evidance do you have, do you have a memory dump, screenshot, can it be repeated? only THEN will they have a fighting chance of fixing it.
everything else is just noise and supposition until proved otherwise.
the only thing i can see they are "guilty" of is lack of communications. its my opionon they do need to get better with their comms base. The jobs that @ZOS_GinaBruno and @ZOS_JessicaFolsom and the team do is thankless - and they must get some sort of abuse! id ask people that they remember that its not their personal fault and to pay them due respect.
At the same time and speaking as an end user its my opinion that comms to the user base needs to be done in a "different way" in order to commercially satisfy what is increasingly becoming a disgruntled user base.
PS just my opinion and PS no i dont work for them!. i just have a lot of sympathy being in a senior position in an unrelated company doing a similar thing
im sure theres a lot of factors i haven talked about and happy to chat further with those who may have a different stance. if youve got this far. well done - have some anniversary cake.