Home Forums Management Complexity Management Nonergodicity vs. Big Data and Predictive Analytics

Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • Andrey ShovkoplyasAndrey Shovkoplyas
    Member
    Post count: 2
    #12498 |

    Joss,

    Not being an expert in the field, I very much agree with the way how you’ve explained the issue of ergodicity and nonergodicity in your tutorial and what it means for our ability to predict a future state of a system. I thought that anyone who is familiar with a concept of a phase transition would agree with its relation to nonergodicity.

    At the same time, everyone can observe the rapid advancement of Big Data and predictive analytics, which seems to “promise” just the opposite of what nonergodicity postulates. I’d be very interested in learning you view on this.

    In a broader sense, do you think that the rise of the Big data and analytics makes complex open systems more predictable? From a more philosophical point of view, referring to the Leibnitz-Newton debate, do you think with advances in the big data, analytics and computing capacity (singularity?) we indeed approach a day when we’ll be able to “know and predict position of every atom in the Universe”?

    Thank you

    Andrey

    P.S.:  I’m equally interested to learn other Community members’ opinion on this.

    J. ColchesterJ. Colchester
    Keymaster
    Post count: 5

    Hi Andrey

    Yes, I find it a very interesting topic also. I think it is important to identify that some things are simple and some things are complex and the set of ideas supporting ergodicity only really hold within simpler linear systems. That is to say, they are closed systems, with low levels of feedback so you don’t get emergent processes where the system and its environment coevolve through feedback dynamics that can not be predicted – that only take place as the process plays out and thus can not be fully known at the inception of the process. In closed linear systems, the system has a finite amount of states and given enough samples one can know all of those states, thus it is possible to take a sample from the past and project it on to the future to infer something about the future state. But in complex systems, you get feedback, coevolution, phase transitions and emergence and a sample from the past may not be able to tell you very much about anything on the other side of a phase transition because the whole context has changed.

    I think that big data and advanced analytics will be important tools in managing the ever-growing complexity of our economies and technology infrastructure. But these technologies are only as good as the awareness and intelligence that goes into their making. The risk is of replacing human intuition and intelligence with a very narrow form of automated intelegence that operates and makes decisions within complex environments without all the parameters factored into the algorithms – a good example of this is algorithmic trading, where the algorithms incorporate only a limited number of factors without understanding how they might interact with each other or how the might operate under “non-normal” market conditions. I think big data analytics will be important in many areas – that involve closed analytical reasoning – but certainly not in more complex environments that require one to understand the overall context.

    With respect to the philosophical implications of this, the idea is explored most fully in the discussion surrounding strong and weak emergence. Where weak emergence means that given enough computational capabilities one could fully predict events in the future while strongly emergence phenomena and processes can not in theory or practice be computed (see this video https://goo.gl/FqsfJP) The Leibnitz-Newton debate really comes down to whether the world really is strongly emergent and that is a much-debated issue in philosophy and science, it is a central part of the debate surrounding reductionism and holism, where reductionism holds that the world is weakly emergent and thus could theoretically be computed while holism sees the world as strongly emergent and thus the future can not be fully computed. If you would like to dig into it I would recommend this video series https://goo.gl/KJqMQy

    • This reply was modified 5 days, 14 hours ago by J. Colchester J. Colchester.
    Andrey ShovkoplyasAndrey Shovkoplyas
    Member
    Post count: 2

    Joss – Thank you for your answer. Makes a lot of sense. Love watching Kauffman!
    Best,
    Andrey

Viewing 3 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.