Mihir Sarkar

What does it take for a utility company to develop a data- and AI-driven software business?

Music has been shown to form an essential part of the human experience-every known society engages in music. However, as universal as it may be, music has evolved into a variety of genres, peculiar to particular cultures. In fact people acquire musical skill, understanding, and appreciation specific to the music they have been exposed to. This process of enculturation builds mental structures that form the cognitive basis for musical expectation. In this thesis I argue that in order for… Show more

Music has been shown to form an essential part of the human experience-every known society engages in music. However, as universal as it may be, music has evolved into a variety of genres, peculiar to particular cultures. In fact people acquire musical skill, understanding, and appreciation specific to the music they have been exposed to. This process of enculturation builds mental structures that form the cognitive basis for musical expectation. In this thesis I argue that in order for machines to perform musical tasks like humans do, in particular to predict music, they need to be subjected to a similar enculturation process by design. This work is grounded in an information theoretic framework that takes cultural context into account. I introduce a measure of musical entropy to analyze the predictability of musical events as a function of prior musical exposure. Then I discuss computational models for music representation that are informed by genre-specific containers for musical elements like notes. Finally I propose a software framework for automatic music prediction. The system extracts a lexicon of melodic, or timbral, and rhythmic primitives from audio, and generates a hierarchical grammar to represent the structure of a particular musical form. To improve prediction accuracy, context can be switched with cultural plug-ins that are designed for specific musical instruments and genres. In listening experiments involving music synthesis a culture-specific design fares significantly better than a culture-agnostic one. Hence my findings support the importance of computational enculturation for automatic music prediction. Furthermore I suggest that in order to sustain and cultivate the diversity of musical traditions around the world it is indispensable that we design culturally sensitive music technology. Show less

Listening to music is not only conditioned by our perception of structured auditory stimuli, but also by our cultural background: someone who grows up listening only to Western classical music is unlikely to fully appreciate contemporary popular music, let alone understand the ragas and talas of Carnatic or Hindustani music. To study the role of informal enculturation and formal education on our musical mind, I have developed several computational methods. These tools not only provide us with… Show more

Listening to music is not only conditioned by our perception of structured auditory stimuli, but also by our cultural background: someone who grows up listening only to Western classical music is unlikely to fully appreciate contemporary popular music, let alone understand the ragas and talas of Carnatic or Hindustani music. To study the role of informal enculturation and formal education on our musical mind, I have developed several computational methods. These tools not only provide us with quantitative analyses and means to compare musical styles and genres, but also with a framework to develop new computer-based applications to enhance our musical abilities in a variety of dimensions, i.e. timbre, rhythm, and melody, and in a range of musical traditions (unlike current music technology, which caters primarily to Western music). Some of these systems are based on a structured audio language derived from Csound, which was developed by Prof. Barry Vercoe of the MIT Media Lab, and which is at the core of MPEG-4's audio standard. Some of the applications that are enabled by our research include:
1. A network music performance system for the tabla where musicians located at a distance can play together in a live performance.
2. A synthesizer that generates sounds based on words like "warm" and "bright" instead of numerical parameters.
3. An interface to compose melodies taking into account the gamakas of Indian music.

I believe this is a step toward preserving as well as enhancing non-Western musical traditions. Show less

Do you remember asking your bass player — if you ever played in a band — to make his sound fatter, or even describing your LP albums as warmer than their CD version? In fact, musicians and listeners often describe sound with words borrowed from a variety of domains, including vision (bright), touch (rough), and material properties (metallic). However, in our experience with musicians we noticed that brass players, for instance, used a specific vocabulary suited to their instrument, and… Show more

Do you remember asking your bass player — if you ever played in a band — to make his sound fatter, or even describing your LP albums as warmer than their CD version? In fact, musicians and listeners often describe sound with words borrowed from a variety of domains, including vision (bright), touch (rough), and material properties (metallic). However, in our experience with musicians we noticed that brass players, for instance, used a specific vocabulary suited to their instrument, and different from, say, singers or violinists. We therefore postulated that the words used to describe sound were dependent on a persons musical, and possibly cultural, background.

To verify our hypothesis, we designed and administered an online survey in which participants were asked to describe sounds that were presented to them by either selecting words from a list, describing them with their own words, or comparing two sounds (this sound is sharper than that one). We used 64 sound files that were gathered from Freesound, a user-generated online sound database, and combined them randomly with a list of 62 words that we collected from a prior survey and literature searches. 844 subjects from various countries and musical backgrounds took our survey. Each person was assigned to one or more musical categories (strings, woodwinds, electronic, percussion, and brass), and the results were compared across categories. Statistical measures were employed to determine the relationship between musical background and survey responses.

Surprisingly, our study indicated that people tend to use similar words to describe a particular sound. We therefore concluded that the description of musical sounds is a universal skill that is not influenced by musical training and preference. This sheds a different light on how our brain processes sound quality or timbre, and could lead to a self-reflection tool in the education of musicians and sound engineers. Show less

Thanks to the Internet, musicians located in different countries can now aspire to play with each other almost as if they were in the same room. However, the time delays due to the inherent latency in computer networks (up to several hundreds of milliseconds over long distances) are unsuitable for musical applications. Some musical collaboration systems address this issue by transmitting compressed audio streams (such as MP3) over low-latency and high-bandwidth networks (e.g. LANs or Internet2)… Show more

Thanks to the Internet, musicians located in different countries can now aspire to play with each other almost as if they were in the same room. However, the time delays due to the inherent latency in computer networks (up to several hundreds of milliseconds over long distances) are unsuitable for musical applications. Some musical collaboration systems address this issue by transmitting compressed audio streams (such as MP3) over low-latency and high-bandwidth networks (e.g. LANs or Internet2) to constrain time delays and optimize musician synchronization. Other systems, on the contrary, increase time delays to a musically-relevant value like one phrase, or one chord progression cycle, and then play it in a loop, thereby constraining the music being performed. In this thesis I propose TablaNet, a real-time online musical collaboration system for the tabla, a pair of North Indian hand drums. This system is based on a novel approach that combines machine listening and machine learning. Trained for a particular instrument, here the tabla, the system recognizes individual drum strokes played by the musician and sends them as symbols over the network. A computer at the receiving end identifies the musical structure from the incoming sequence of symbols by mapping them dynamically to known musical constructs. To deal with transmission delays, the receiver predicts the next events by analyzing previous patterns before receiving the original events, and synthesizes an audio output estimate with the appropriate timing. Although prediction approximations may result in a slightly different musical experience at both ends, we find that this system demonstrates a fair level of playability by tabla players of various levels, and functions well as an educational tool. Show less

Playing music over the Internet, whether for real-time jamming, distributed performance or distance education, is constrained by the network latency which introduces, over long distances, time delays unsuitable for musical applications. Current musical collaboration systems generally transmit compressed audio streams over low-latency and high-bandwidth networks to optimize musician synchronization. This paper proposes an alternative approach based on pattern recognition and music prediction… Show more

Playing music over the Internet, whether for real-time jamming, distributed performance or distance education, is constrained by the network latency which introduces, over long distances, time delays unsuitable for musical applications. Current musical collaboration systems generally transmit compressed audio streams over low-latency and high-bandwidth networks to optimize musician synchronization. This paper proposes an alternative approach based on pattern recognition and music prediction. Trained for a particular type of music, here the Indian tabla drum, the system called TablaNet identifies rhythmic patterns by recognizing individual strokes played by a musician and mapping them dynamically to known musical constructs. Symbols representing these musical structures are sent over the network to a corresponding computer system. The computer at the receiving end anticipates incoming events by analyzing previous phrases and synthesizes an estimated audio output. Although such a system may introduce variants due to prediction approximations, resulting in a slightly different musical experience at both ends, we find that it demonstrates a high level of playability with an immediacy not present in other systems, and functions well as an educational tool. Show less

Most Digital Signal Processors perform computations on integers, or fixed-point numbers, rather than floating-point numbers. In contrast, Digital Signal Processing algorithms are often designed with real numbers in mind and usually implemented in floating-point. Apart from finite word length effects that may appear during signal acquisition and intermediate computations, limits on the signal precision and range often compromise the stability of the system.

Audio algorithms are… Show more

Most Digital Signal Processors perform computations on integers, or fixed-point numbers, rather than floating-point numbers. In contrast, Digital Signal Processing algorithms are often designed with real numbers in mind and usually implemented in floating-point. Apart from finite word length effects that may appear during signal acquisition and intermediate computations, limits on the signal precision and range often compromise the stability of the system.

Audio algorithms are particularly sensitive to fixed-point implementations due to the audible artifacts that the conversion process may introduce. Therefore, it is essential to validate the stability and static characteristics of the system after conversion. Then, the dynamic behavior of the system can be studied by applying suitable test signals.

Starting with a presentation of basic Digital Signal Processing concepts relevant to our discussion, this paper carries on with a floating-point to fixed-point conversion strategy for audio processing algorithms. Finally, a high-pass IIR filter implementation example is presented. Show less