# Stellar Exploratory Data Analysis or How to create the HR Diagram with R

I recently have started to refresh my skills with R programming language. I am doing the  Harvard Course on Data Science on EdX. I am using R Studio for doing all the exercises. In the second part of the course, Visualisation, which is an area of research interest for me, there is an exercise on stars dataset. But this exercise was available only to those who were crediting the course. Since I was not crediting, but only auditing I left the exercise as it is. But after a week or so I looked at the stars dataset. And thought I should do some explorations on this. For this we have to load the R package dslabs specially designed for this course. This post is detailing the exploratory data analysis with this dataset. (Disclaimer: I have used help from ChatGPT in writing this post for both content and code.)

> library(dslabs)

Once this is loaded, we load the stars dataset

data(stars)

## Structure of the dataset

To understand what is the data contained in this data set and how is it structured we can use several ways. The head(stars) command will give use first few lines of the data set.

> head(stars) star magnitude temp type 1 Sun 4.8 5840 G 2 SiriusA 1.4 9620 A 3 Canopus -3.1 7400 F 4 Arcturus -0.4 4590 K 5 AlphaCentauriA 4.3 5840 G 6 Vega 0.5 9900 A

While the  tail(stars) gives last few lines of the data set

tail(stars) star magnitude temp type 91 *40EridaniA 6.0 4900 K 92 *40EridaniB 11.1 10000 DA 93 *40EridaniC 12.8 2940 M 94 *70OphiuchiA 5.8 4950 K 95 *70OphiuchiB 7.5 3870 K 96 EVLacertae 11.7 2800 M

To understand structure further we can use the str(stars) command

> str(stars) 'data.frame': 96 obs. of 4 variables: $star : Factor w/ 95 levels "*40EridaniA",..: 87 85 48 38 33 92 49 79 77 47 ...$ magnitude: num 4.8 1.4 -3.1 -0.4 4.3 0.5 -0.6 -7.2 2.6 -5.7 ... $temp : int 5840 9620 7400 4590 5840 9900 5150 12140 6580 3200 ...$ type : chr "G" "A" "F" "K" ...

In RStudio we can also see the data with View(Stars) function in a much nicer (tabular) way. It opens up the data in another frame as shown below.

Thus we see that it has 96 observations with four variables, namely star, magnitude, temp and type. The str(stars) command also tells use the datatype of the columns, they are all different: factor, num, int, chr. Let us understand what each of the column represents.

## Name of stars

The star variable has the names of the stars as seen in the table above. Many of the names are of ancient and mythological origins, while some are modern. Most are of Arabic origin, while few are from Latin. Have a look at Star Lore of All Ages by William Olcott to know some of the mythologies associated with these names. Typically the alphabets after the star names indicate them being part of a stellar system, for example Alpha Centauri is a triple star system. The nomenclature is such that A represents the brightest member of the system, B the second brightest and so on. Also notice that some names have Greek pre-fixes, as in the case of of Alpha Centauri. This Greek letter scheme was introduced by Bayer in 1603 and is known as Bayer Designation. The Greek letters  denote the visual magnitude or brightness (we will come to the meaning of this next) of the stars in a given constellation. So Alpha Centuari would mean the brightest star in the Centaurus constellation. Before invention of the telescope the number of stars that are observable were limited by the limits of human visual magnitude which is about +6. With invention of telescope and their continuous evolution with increasing light gathering power, we discovered more and more stars. Galileo is the first one to view new stars and publish them in his Sidereal Messenger. He shows us that seen through the telescope, there are many more stars in the Pleiades constellation than can be seen via naked eyes (~+6 to max +7 with about 4200 stars possibly visible).

Soon, so many new stars were discovered that it was not possible to name them all. So coding of the names begun. The large telescopes which were constructed would do a sweep of the sky using big and powerful lenses and would create catalogue of stars. Some of the names in the data set indicate these data sets, for example HD denotes Henry Draper Catalogue.

## Magnitudes of stars

Now let us look at the other three columns present us with observations of these stars. Let us understand what they mean. The second column represents magnitude of the stars. The stellar magnitude is of two types: apparent and absolute. The apparent magnitude is a measure of the brightness of the star and depends on its actual brightness, distance from us and any loss of the brightness due to intervening media. The magnitude scale was devised by Claudius Ptolemy in second century. The first magnitude stars were the brightest in the sky with sixth being the dimmest. The modern scale follows this classification and has made it mathematical. The scale is reverse logarithmic, meaning that lower the magnitude, brighter is the object. A magnitude difference of 1.0 corresponds to a brightness ratio of $\sqrt[5]{100}$ or about 2.512. Now if you are wondering why the magnitude scale is logarithmic, the answer lies in the physiology of our visual system. As with the auditory system, our visual system is not linear but logarithmic. What this means is that if we perceive an object to be of double brightness of another object, then their actual brightness (as measured by a photometer) are about 2.5. This fact is encapsulated well in the Weber-Fechnar law. The apparent magnitude of the Sun is about -26.7, it is after all the brightest object in the sky for us. Venus, when it is brightest is about -4.9. The apparent magnitude of Neptune is +7.7 which explains why it was undiscovered till the invention of the telescope.

But looking at the table about the very first entry lists Sun’s magnitude as +4.8. This is because the dataset contains the absolute magnitude and not the apparent magnitude. Absolute magnitude is defined as “apparent magnitude that the object would have if it were viewed from a distance of exactly 10 parsecs (32.6 light-years), without dimming by interstellar matter and cosmic dust.” As we know, the brightness of an object is inversely proportional to square of the distance (inverse square law). Due to this fact very bright objects can appear very dim if they are very far away, and vice versa. Thus if we place the Sun at a distance of about 32.6 light years it will be not-so-bright and will be an “average” star with magnitude +4.8. The difference in these two magnitudes is -31.57 and this translates to huge brightness difference of 3.839 $\times$ 1012. And of course this  definition does not take into account the interstellar matter which further dims the stars. Thus to find the absolute magnitude of the stars we also need to know their distance. This is possible for some nearby stars for which the parallax has been detected. But for a vast majority of stars, the parallax is too small to be detected because they are too faraway. The distance measure parsec we saw earlier is defined on basis of parallax, one parsec is the distance at which 1 AU (astronomical unit: distance between Earth and Sun) subtends an angle of one arcsecond or 1/3600 of a degree.

Thus finding distance to the stars is crucial if we want to know their actual magnitudes. For finding the cosmic distances various techniques are used, we will not go into their details. But for our current purpose, we know that the stars dataset has absolute magnitudes of stars. The range of magnitudes in the dataset is

> range(stars$magnitude) [1] -8 17 Thus stars in the dataset have a difference of 25 magnitudes, that is a brightness ratio of 105! Which are these brightest and dimmest stars? And how many stars of each magnitude are there in the data set? We can answer these type of questions with simple queries to our dataset. For starters let us find out the brightest and dimmest stars in the dataset. Each row in the dataset has an index, which is the first column in the table from RStudio above. Thus if we were to write: > stars[1] it will give us all the entries of the first column, star 1 Sun 2 SiriusA 3 Canopus 4 Arcturus 5 AlphaCentauriA 6 Vega 7 Capella 8 Rigel 9 ProcyonA 10 Betelgeuse ... ... But if we want only a single row, instead of a column, we have to tell that by keeping a , in the index 1. Thus for the first row we write  > stars[1,] > star magnitude temp type 1 Sun 4.8 5840 G Thus to find the brightest or dimmest star we will have to find its index and then we can find its name from the corresponding column. So how do we do that? For this we have functions which.max and which.min, we use them thus: > which.max(stars$magnitude) [1] 76

We feed this to the dataset and get
 > stars[76,] star magnitude temp type 76 G51-I5 17 2500 M

This can also be done in a single line

> stars[which.min(stars$magnitude), ] star magnitude temp type 45 DeltaCanisMajoris -8 6100 F Now let us check the distribution of these magnitudes. The simplest way to do this is to create a histogram using the hist function. hist(stars$magnitude)

This gives the following output

As we can see it has by default binned the magnitudes in bins of 5 units and the distribution here is bimodal with one peak between -5 and 0 and another peak between 10 and 15. We can tweak the width of the bars to get a much finer picture of the distribution. For this hist function has option to add breaks manually. We have used the seq function here ranging from -10 to 20 in steps of 1.

> hist(stars$magnitude, breaks = seq(-10, 20, by = 1)) And this gives us: Thus we see that the maximum number of stars (9) are at -1 magnitude and three magnitudes have one star each while +3 magnitude doesn’t have any stars. This histogram could be made more reader friendly if we can add the count on the bars. For this we need to get some coordinates and numbers. We first get the counts mag_data <- hist(stars$magnitude, breaks = seq(-10,20, 1), plot = FALSE)

This will give us the actual number of counts

> [1] 0 1 2 1 7 6 4 3 3 9 6 4 4 0 2 5 2 2 2 1 5 7 3 7 5 3 2 0 0 0

Now to place them at the middle of the bars of histogram we need midpoints of the bars, we use mag_data$mids to find them and mag_data_counts for the count for labels. > text(mag_data$mids, mag_data$counts, labels = mag_data$counts, pos = 3, cex = 0.8, col = "black")

To get the desired graph

Thus we have a fairly large distribution of stellar magnitudes.

Now if we ask ourselves this question How many stars in this dataset are visible to the naked eye? What can we say? We know that limiting magnitude for naked eye is +6. So, a simple query should suffice:

count(stars %>% filter(magnitude <= 6)) n 1 57

(Here we have used the pipe function  %>%  to pass on data from one argument to another from the dplyr pacakge. This query shows that we have 57 stars which have magnitude less than or equal to 6. Hence these many should be visible… But wait it is the absolute magnitude that we have in this dataset, so this question itself cannot be answered unless we have the apparent magnitudes of the stars. Though computationally correct, this answer has no meaning as it is cannot be treated same as the one with apparent magnitude which we experience while watching the stars.

## Temperature of Stars

The third column in the data set is the temp or the temperature. Now, at one point in the history of astronomy people believed that we would never be able to understand the structure or the content of the stars. But the invention of spectroscopy as a discipline and its application to astronomy made this possible. With the spectroscope applied to the end of the telescope (astronomical spectroscopy), we could now understand the composition of the stars, their speed and their temperature. The information for the composition came from the various emission and absorption lines in the spectra of the stars, which were then compared with similar lines produced in the laboratory by heating various elements. Helium was first discovered in this manner: first in the spectrum of the Sun and then in the laboratory. For detailed story of stellar spectroscopy one can see the book Astronomical Spectrographs and Their History by John Hearnshaw. Though an exact understanding of the origin of the spectral line came only after the advent of quantum mechanics in early part of 20th century.

But the spectrum also tells us about the surface temperature of the stars. How this is so? For this we need to invoke one of the fundamental ideas in physics: the blackbody radiation. Now if we find the intensity of radiation from a body at different wavelengths (or frequencies) we get a curve. This curve is typical and for different temperatures we get unique curves (they don’t intersect). Of course this is true for an ideal blackbody which is an idealized opaque, non-reflective body. Stellar spectrum is like that of an ideal blackbody,  this continuous spectrum is punctuated with absorption and emission lines as shown in the book cover above.

The frequency or wavelength at which the radiation has maximum intensity (brightness/luminosity) is related to the temperature of the body, typical curves are shown as above. Stars behave almost as ideal black bodies. Notice that as the temperature of the body increases the peak radiation wavelength increases (frequency is reduced) as shown in the diagram above. These relationships are given by the formula

$$L = 4 \pi R^{2} \sigma T^{4}$$

where $L$ is the luminosity, $R$ is the radius, $\sigma$ is Stephan’s constant and $T$ is the temperature. This equation tells us that $L$ is much more dependent on the $T$, so hotter stars would be more brighter.

It was failure of the classical ideas of radiation and thermodynamics to explain the nature of blackbody radiation that led to formulation of quantum mechanics by Max Planck in the form of Planck’s law for quantisation of energy. For a detailed look at the history of this path breaking episode in history of science one of the classics is Thomas Kuhn’s Black-Body Theory and the Quantum Discontinuity, 1894—1912.

That is to say hotter bodies have shorter peak frequencies. In other words, blue stars are hotter than the red ones. (Our hot and cold symbolic colours on the plumbing peripherals needs to change: we have it completely wrong!) Thus the spectrum of the stars gives as its absolute temperature, along with all other information that we can obtain from the stars. The spectrum is our only source of information for stars. This is what is represented in the third column of our data. For our dataset the range of stellar temperatures we have a wide range of temperatures.

range(stars$temp) [1] 2500 33600 Let us explore this column a bit. If we plot a histogram with default options we get: > hist(stars$temp)

This is showing maximum stars have a temperature below 10000. We can bin at 1000 and add labels to get a much better sense. Which star has 0 temperature??

hist(stars$temp, breaks = seq(0,35000, 1000)) > temp_data <- hist(stars$temp, breaks = seq(0,35000, 1000), plot = FALSE) > text(temp_data$mids, temp_data$counts, labels = temp_data$counts, pos = 3, cex = 0.8, col = "black") This plot gives us much better sense of the distribution of stellar temperatures. With most of the temperatures being in 2000-3000 degrees Kelvin range. The table()  function also provides useful information about distribution of temperatures in the column. > table(stars$temp)

2500 2670 2800 2940 3070 3200 3340 3480 3750 3870 4130 4590 1 10 7 5 1 3 4 1 1 2 3 3 4730 4900 4950 5150 5840 6100 6580 6600 7400 7700 8060 9060 1 5 1 2 2 2 1 1 2 1 2 1 9300 9340 9620 9700 9900 10000 11000 12140 12400 13000 13260 14800 1 2 3 1 4 1 1 1 1 1 1 1 15550 20500 23000 25500 26950 28000 33600 1 4 2 5 1 2 1

While the summary() function provides the basic statistics:

> summary(stars$temp) Min. 1st Qu. Median Mean 3rd Qu. Max. 2500 3168 5050 8752 9900 33600    ## Type of Stars The fourth and final column of our data is type. This category of data is again based on the spectral data of stars and is type of spectral classification of stars. “The spectral class of a star is a short code primarily summarizing the ionization state, giving an objective measure of the photosphere’s temperature. ” The categories of the type of stars and their physical properties are summarised in the table below. The type of stars and their temperature is related, with “O” type stars being the hottest, while “M” type stars are the coolest. The Sun is an average “G” type star. There are several mnemonics that can help one remember the ordering of the stars in this classification. One that I still remember from by Astrophysics class is Oh Be A Fine Girl/Guy Kiss Me Right Now. Also notice that this “type” classification is also related to size of the stars in terms of solar radius. In our dataset, we can see what type of stars we have by > stars$type [1] "G" "A" "F" "K" "G" "A" "G" "B" "F" "M" "B" "B" "A" "K" [15] "B" "M" "A" "K" "A" "B" "B" "B" "B" "B" "B" "A" "M" "B" [29] "K" "B" "A" "B" "B" "F" "O" "K" "A" "B" "B" "F" "K" "B" [43] "B" "K" "F" "A" "A" "F" "B" "A" "M" "K" "M" "M" "M" "M" [57] "M" "A" "DA" "M" "M" "K" "M" "M" "M" "M" "K" "K" "K" "M" [71] "M" "G" "F" "DF" "M" "M" "M" "M" "K" "M" "M" "M" "M" "M" [85] "M" "DB" "M" "M" "A" "M" "K" "DA" "M" "K" "K" "M"

Our Sun is G-type star in this classification (first entry). If we use the table() function on this column we get the frequency of each type of star in the dataset.

> table(stars$type) A B DA DB DF F G K M O 13 19 2 1 1 7 4 16 32 1 And to see a barplot of this table we will use ggplot2() package. Load the package using library using library(ggplot2) and then > stars %>% ggplot(aes(type)) + geom_bar() + geom_text(stat = "count", aes(label = after_stat(count)), vjust = -0.5, size = 4) Thus we see that “M” type stars are the maximum in our dataset. But we can do better, we can sort this data according the frequency of the types. For this we use the code: > type_count <- table(stars$type) > # count the frequencies > sorted_type <- names(sort(type_count)) > # sort them > stars$type <- factor(stars$type, levels = sorted_type) > # reorder them with levels and plot them > stars %>% ggplot(aes(type)) + geom_bar(fill = "darkgray") + geom_text(stat = "count", aes(label = after_stat(count)), vjust = -0.5, size = 4)

And we get

## To plot HR Diagram

Now, given my training in astronomy and astrophysics, the first reaction that came to my mind after seeing this data was this is the data for the HR Diagram! The HR diagram presents us with the fundamental relationship of types and temperature of stars. This was an crucial step in understanding stellar evolution. The intials HR stand for the two astronomers who independently found this relationship: The diagram was created independently in 1911 by Ejnar Hertzsprung and by Henry Norris Russell in 1913.

By early part of 20th century several star catalogues had been around, but nothing stellar evolution or structure was known. The stellar spectrographs revealed what elements were present in the stars, but the energy source of the stars was still an unresolved question. Classical physics had no answer to this fundamental question about how stars were able to create so much energy (for example, see Stars A Very Short Introduction by James Kaler on the idea that charcoal powers the Sun by Lord Kelvin). Added to this was the age of the stars, from geological data and idea of geological deep time, the Sun was estimated to be 4 billion years old as was the Earth. So stars had been producing so much energy for such a long time! But that is not the point of this post, the HR diagram definitely helped the astronomers think about the idea that stars might not be static but evolve in time. The International Astronomical Union conducted a special symposium titled The HR Diagram in 1977. The proceedings of the symposium have several articles of interest on the history of creation and interpretation of the HR Diagram.

I think it was but natural that astronomers tried to find correlations between various properties of thousands of stars in these catalogues. And when they did they find a (co-)relationship between them. The HR diagram exists in many versions, but the basic idea is to plot the absolute magnitude and temperature (or colour index). Let us plot these two  to see the co-relation, for this we again use the ggplot2() pacakge and its scatterplot function geom_point().

> stars %>% ggplot(aes(temp, magnitude)) + geom_point()

This gives us the basic plot of HR diagram.

Immediately we can see that the stars are not randomly scattered on this plot, but are grouped in clusters. And most of them lie in a “band”. Though there are outliers at the lower temperature and magnitude range and high magnitude and temperature around 10-15 thousand range. We see that most stars lie in a band which is called the “Main Sequence”. We can try to fit a function here in this plot using some options in the ggplot() library, we use geom_smooth() function for this and get:

stars %>% ggplot(aes(temp, magnitude)) + geom_point() + geom_smooth( se = FALSE, color = “red”)

Of course this smooth curve is a very crude (perhaps wrong?) approximation of the data, but it certainly points us towards some sort of correlation between the two quantities for most of the stars. But wait, we have another categorical variable in our dataset, the type of stars. How are the different types of stars distributed on this curve? For this we introduce type variable in the aesthetics argument of ggplot() to colour the stars on our plot according to this category:

> stars %>% ggplot(aes(temp, magnitude, color = type)) + geom_smooth( se = FALSE, color = "red") + geom_point()

This produces the plot

Thus we see there is a grouping of stars by the type. Of course the colours in the palette here are not the true representatives of the star colours. The HR diagram was first published around 1911-13, when quantum mechanics was in its nascent stages. The ideas of Rutherford’s model were still extant and was just out. The fact that this diagram indicated a relationship between the magnitude and temperature, led to thinking about stellar structure itself and its ways of producing energy with fundamentally new ideas about matter and energy from quantum mechanics and their transformation from relativistic physics. But that is a story in future. For now, let us come to our HR diagram. From the dataset we have one more variable, the star name which could be used in this plot. We can name all the stars in the plot (there are only 96). For this we use the geom_text() function in ggplot()

> stars %>% ggplot(aes(temp, magnitude, color = type), label = star) + geom_smooth( se = FALSE, color = "red") + geom_point() + geom_text((aes( label = star)), nudge_y = 0.5, size = 3)

This produces a rather messy plot, where most of the starnames are on top of each other and not readable:

To overcome this clutter we use another package ggrepel() with the following code:

> stars %>% ggplot(aes(temp, magnitude, color = type), label = star) + geom_smooth( se = FALSE, color = "red") + geom_text_repel(aes(label = star))

This produces the plot with the warning "Warning message: ggrepel: 13 unlabeled data points (too many overlaps). Consider increasing max.overlaps ". To overcome this we increase the max.overlaps to 50.

> stars %>% ggplot(aes(temp, magnitude, color = type), label = star) + geom_point() + geom_smooth( se = FALSE, color = "red") + geom_text_repel(aes(label = star), max.overlaps = 50)

This still appears cluttered a bit, scaling the plot while exporting gives this plot, though one would need to zoom in to read the labels.

Of course with a different data set, with larger number and type of stars we would see slightly different clustering, but the general pattern is the same.

We thus see that starting from the basic data wrangling we can generate one of the most important diagrams in astrophysics. I learned a lot of R in the process of creating this diagram. Next task is to

# Obedience and Conformity

Obedience can be understood as “a compliance with an order, request, or law or submission to another’s authority”, while one of the meanings of conformity is “behaviour in accordance with socially accepted convention“. Obedience to authority forms one of the basis of civilisation. Unless the citizens abide by some rules there will be a complete chaos. In early days, this was done by rulers in form of rules and edicts. Now we have the constitution.

Obedience is essential and must be achieved by cunning…

We usually use obedience and conformity as interchangeable terms or having at least similar connotations. but they are not same. But both are forms of power and control over other humans. Here is how Orwell puts it forth in Nineteen Eighty Four.

“The real power, the power we have to fight for night and day, is not power over things, but over men.” [O’Brien] paused, and for a moment assumed again his air of a schoolmaster questioning a promising pupil:

“How does one man assert his power over another, Winston?”

Winston thought. “By making him suffer,” he said.

“Exactly. By making him suffer. Obedience is not enough. Unless he is suffering, how can you be sure that he is obeying your will and not his own? Power is in inflicting pain and humiliation. Power is in tearing human minds to pieces and putting them together again in new shapes of your own choosing.”

Stanley Milgram made a distinction between the two and identified four major differences that distinguish the two. This differences tell us about the nature of obedience and conformity. The core of the difference lies in the person who is commanding obedience and the person who is following that in case of conformity usually the person following others is at equal status to them. Where is in the case of obedience there is a hierarchy between the commander and the commanded. This is the first difference: the hierarchy. In a classroom the teacher commands the students to follow certain task or to do some task. This is an example of obedience. When students try to follow each other in terms of the things they do or they dress outside of school or the games that play or the clothes they wear is closer to conformity. The in the first case is the clear distinction and difference in hierarchy in the second case it is not so.

The second major difference is imitation while obedience involves compliance with orders, confirmatory involves imitation and adoption of similar behaviour. If neighbours are beating the plates to make corona go away, so shall I, otherwise we will be seen as socially non-fitting.

The third difference is that in case of obedience the commands are more explicit whereas for conformity is more of an implicit thing. An unspoken rule which everybody around you follows. Conformity is more about following others who are your peers than following explicit commands so in case of confirmative their might be some elbow room to express yourself slightly differently than others. But in case of obedience that is not possible obedience is to the full and no lateral thinking is allowed.

Final difference is the voluntary nature of conformity as opposite to attribution to authority figures. In case of obedience there is no choice but to follow the orders.

Milgram’s experiment on obedience shows how suggestible humans are in presence of authority.

Would you torture another human because someone in authority tells you to?

The experiment is setup as such. The subject let us call them X is called for doing an experiment in the lab. X is then introduced to the experiment. The experiment involves X teaching another participant Y word association using punishment for wrong answers. If Y does not give correct answer, X has to give Y a punishment in the form of an electric shock. X is given a mild shock to make them experience of the punishment, so that X knows what is the type of punishment and how Y would feel when punished. X has a dial which can control the amount of shock delivered to Y. Overseeing all this is a researcher Z in a white lab coat. A person wearing a white lab coat somehow represents authority figure for most. This is why you see doctors/scientists in advertisements wearing white lab coat: because it is a symbol of authority

Now, Y is actually part of the experiment and accomplice of the researcher. X does not know this. The test starts, and Y deliberately chooses to give wrong answers. Now according to the “rules” of the experiment X has to give “punishment” to Y for incorrect answers. The dial for controlling the shock, the “shock generator”, is calibrated with incremental levels of shock, finally going to 450 Volts. (Of course this is make believe, there is no real shock given, but let us keep this secret with us and not tell X.)

With each wrong answer X is supposed to increment the level of shock punishment to Y from the shock generator. Now, X knows that increasing the shock will cause incremental pain to Y. When X hesitates to increase, Z intervenes and tells X authoritatively (remember Z is wearing a white lab coat) that this is the rule: “With each wrong answer punishment must be increased.” This is where the crux of the experiment comes in. It is found that majority of X, under the influence from authority figure Z, obediently inflicts serious punishment to Y. This even when Y cries in agony, asks experiment to stop, but more X continue to punish Y.

This is highly counterintuitive result. How so you would ask?  Consider the following gedankenexperiment. Suppose someone looking authoritative tells you to go and thrash someone. Would you do it? Most probably the answer would be no. Then why are the participants X so willing to inflict the incrementally harsh punishment? Because they start with something that is seemingly innocent – a mild shock. Once that threshold is crossed rest of the punishment becomes easy. And you adjudge yourself being not at fault as you are “just following the orders or rules”. This has large implications for how we as a society, every now and then, fall for authority figures and do things which we think we will not do. Perhaps in such cases, we think that since authority figure is telling us to do something, it must be for greater good and we let not our puny morals or conscience come in the way. Also, since we are obeying authority, it takes off our own personal responsibilities as such. This is a slippery slope and can lead to genocides and living hell for those Y who are at the suffering end. Just because Z is popular and in authority does not make following them blindly a right thing. But then the aspect of conformity sets in. If we don’t follow the current norms we are seen as outcasts in the society and that leads to further pressuring of conformity.

Moral is we should exercise our own set of morals and conscience as much as possible.

Sources

Great Ideas in Psychology by Fathali Moghaddam

Nineteen Eighty Four by George Orwell

# Nominal expertise

Who is an expert? What qualities in a person defines them as an expert? The dictionary meaning of an expert is

a person who is very knowledgeable about or skilful in a particular area

But how do we know if a person is knowledgeable or skilful in a particular area? A simple way to answer this question is “expert is one who has expertise!”. But this really does not tell us anything (or it does?) about the nature of expertise. One better way to characterise expertise would be if we know by some objective manner that a particular person is an expert. One such way can be to look at the educational qualifications of the person under question. For example, if someone says “You can ask her any question about stars. She has a PhD in stellar astrophysics.” you take on authority of the person telling you and the fact of having a PhD that the person is indeed an “expert”. This is because PhD requires detailed study (at least of a part of the subject area) and we assume that people who have this degree also have a sufficient expertise. PhD holders are highly educated is the claim. Hence most of the experts would be PhD holders in their respective fields. But having a PhD is no guarantee that the person indeed is an expert in the field of study. This is what Frederick Reif has to say about it in his article Interpretation of Scientific Concepts:

Quite a few physics graduate students, and even some physics professors, make mistakes and arrive at wrong answers. Indeed, some experts’ performance resembles that of novices. Such observations indicate that nominal experts (i.e., persons designated as “expert” by virtue of their degrees, A. titles, or positions) can differ very widely in their actual competence. (To paraphrase George Orwell, some experts are much more equal than others). This should be a warning about the interpretation of many cognitive studies where “experts” are selected by purely nominal criteria, without specifying adequately the nature of their actual expertise.

This I feel is a case for normative vs. descriptive dichotomy. The position or degree of a person gives them the virtue of being an expert, but it does not guarantee it. And when we decide our policies based on the expertise of the experts which may not be a true expertise or maybe inherently biased. Perhaps this is one of the reasons that we have flawed policies in the first place. Though, Dr. Strangelove (Dr. suggesting a PhD) was an expert!
But are there experts who do not have a PhD or educational qualifications? Yes! Not all knowledge or skills can be concretised in the form of degrees. Most of the knowledge is tacit in nature, which comes from experience. It doesn’t matter if you have a PhD in theoretical hydrodynamics, fixing that leaking tap requires a different type of skill and knowledge. Cooking is another area where knowledge is tacit. Unless you start cooking, you can’t be called an expert cook!

# On respect in the classroom

If you are a teacher (of any sort) and teach young people, don’t be disheartened if the students in your class don’t respect you or listen to you or maintain discipline. Even great philosophers like Socrates and Aristotle has a tough time dealing with their students

Socrates grumbled that he don’t get no respect: his pupils “fail to rise when their elders enter the room. They chatter before company, gobble up dainties at the table, and tyrannize over their teachers.” Aristotle was similarly pissed off by his stu­dents’ attitude: “They regard themselves as omniscient and are positive in their assertions; this is, in fact, the reason for their carrying everything too far.”Their jokes left the philosopher unamused: “They are fond of laughter and conse­quently facetious, facetiousness being disciplined insolence.”
– Judith Harris The Nurture Assumption

That being said, the students are also very perceptive about the knowledge of the teachers, and know who is trying to be a cosmetic intellectual.

# Cosmetic Intellectuals (+ IYI)

In the last few years, the very connotation of the term intellectual has seen a downward slope. Such are the times that we are living in that calling someone an “intellectual” has become more like an insult rather than a compliment: it means an idiot who doesn’t understand or see things clearly. Now as the title of the post suggests it is this meaning, not the other meaning intellectuals who know about cosmetics. Almost two decades back Alan Sokal wrote a book titled Intellectual Impostures, which described quite a few of them. In this book, Sokal exposed the posturing done by people of certain academic disciplines who were attacking science from a radical postmodernist perspective. What Sokal showed convincingly through his famous hoax, is that many of these disciplines are peddling out bullshit with no control over the meaning contained. Only the form was important not the meaning. And in the book, he takes it a step forward, showing that this was not an isolated case. He exposes the misuse of the technical terms (which often have precise and operational meanings) as loose metaphors or even worse completely neglecting the accepted meaning of those terms. The examples given are typical, and you cannot make sense of what is being written. You can read, but cannot understand. It makes no sensible meaning. At this point, you start to doubt your own intelligence and intellectual competence, perhaps you have not read enough to understand this complex piece of knowledge. It was after all written by an intellectual. Perhaps you are not aware of the meaning of the jargon or their context, hence you are not able to understand it. After all there are university departments and journals dedicated to such topics. Does it not legitimise such disciplines as academic and its proponents/followers as intellectuals? Sokal answered it empirically by testing if presented with nonsense whether it makes any difference to the discipline. You are not able to make sense of these texts because they are indeed nonsensical. To expect any semblance of logic and rationality in them is to expect too much.
Nassim Taleb has devised the term Intellectual Yet Idiots (the IYI in the title) in his Incerto series. He minces no words and takes no bullshit. Sokal appears very charitable in comparison. Taleb sets the bar even higher. Sokal made a point to attack mostly the postmodernists, but Taleb bells the cats who by some are even considered proper academics, for example, Richard Dawkins and Steven Pinker. He considers entire disciplines as shams, which are otherwise considered academic, like economics, but has equal if not more disdain to several others also, for example, psychology and gender studies. Taleb has at times extreme views on several issues and he is not afraid to speak of his mind on matters that matter to him. His writings are arrogant, but his content is rigorous and mathematically sound.

they aren’t intelligent enough to define intelligence, hence fall into circularities—their main skill is a capacity to pass exams written by people like them, or to write papers read by people like them.
But there are people who are like IYIs, but don’t even have the depth of the content or knowledge of IYIs. They are wannabe IYIs, all form no conent. They are a level below IYIs. I term such people as cosmetic intellectuals (cosint). We have met them before: they are the envious mediocre and the ones who excel in meetings. The term cosmetic is used in two senses both as adjectives. The first sense is the Loreal/Lakme/Revlon fashion sense as given from the dictionary entry below:

cosmetic

• relating to treatment intended to restore or improve a person’s appearance
• affecting only the appearance of something rather than its substance

It is the second sense that I mean in this post. It is rather the substance of these individuals that is only present in the appearance. And as we know appearance can be deceiving. Cosints appear intellectuals, but only in appearance, hence the term cosmetic. So how does one become a Cosint? Here is a non-exhaustive list that can be an indicator (learn here is not used in the deeper sense of the word, but more like as in rote-learn):

1. Learn the buzzwords: Basically they rote learn the buzzwords or the jargon of the field that they are in. One doesn’t need to understand the deeper significance or meaning of such words, in many cases just knowing the words works. In the case of education, some of these are (non-comprehensive): constructivism, teaching-learning process, milieu, constructivist approaches, behaviorism, classroom setting, 21st-century skills, discovery method, inquiry method, student-centered, blended learning, assessments, holistic, organic, ethnography, pedagogy, curriculum, TLMs. ZPD, TPD, NCF, RTE, (the more complicated the acronyms, the better). More complicated it sounds the better. They learn by association that certain buzzwords have a positive value (for example, constructivism) and other a negative one (for example, behaviorism) in the social spaces where they usually operate in, for example, in education departments of universities and colleges. Not that the Cosints are aware of the deeper meaning of there concepts, still they make a point of using them whenever possible. They make a buzz using the buzzwords. If you ask them about Piaget, they know the very rudimentary stuff, anything deeper and they are like rabbits in front of flashlight. They may talk about p-values, 𝛘2 tests, 98.5 % statistical significances, but when asked will not be able to distinguish between dependent and independent variables.
2. Learn the people: The CosInts are also aware of the names of the people in their trade. And they associate the name to a concept or of a classic work. They are good associating. For example, (bad) behaviorism with Burrhus F. Skinner or Watson, hence Skinner bad. Or Jean Piaget with constructivism and stages (good). Vygotsky: social constructivism, ZPD. Or John Dewey and his work. So they have a list of people and concepts. Gandhi: Nayi Taleem.  Macauley: brought the English academic slavery on India (bad).
3. Learn the classics: They will know by heart all the titles of the relevant classics and some modern ones (you have to appear well-read after all). Here just remembering the names is enough. No one is going to ask you what was said in section 1.2 of Kothari Commission. Similarly, they will rote learn the names of all the books that you are supposed to have read, better still carry a copy of these books and show off in a class. Rote learn a few sentences, and spew it out like a magic trick in front of awestruck students. Items #1 through #3 don’t work very well when they have real intellectual in front of them. A person with a good understanding of basics will immediately discover the fishiness of the facade they put up. But that doesn’t matter most of the time, as we see in the next point.
4. Know the (local) powerful and the famous: This is an absolute must to thrive with these limitations. Elaborated earlier.
5. Learn the language aka Appear academic (literally not metaphorically): There is a stereotype of academic individuals. They will dress in a particular manner (FabIndia?, pyor cotton wonly, put a big Bindi, wear a Bongali kurta etc, carry ethnic items, conference bags (especially the international ones), even conference stationery), carry themselves in a particular manner, talk in a particular manner (academese). This is also true of wannabe CosInt who are still students, they learn to imitate as soon as they enter The Matrix. Somehow they will find ways of using names and concepts from #1 #2 #3 in their talk, even if they are not needed. Show off in front of the students, especially in front of the students. With little practice one can make an entire classroom full of students believe that you are indeed learned, very learned. Any untoward questions should be shooed off, or given so tangential an answer that students are more confused than they were earlier.
6. Attend conferences, seminars and lectures: The primary purpose is network building and making sure that others register you as an academic. Also, make sure that you ask a question or better make a tangential comment after the seminar so that everyone notices you. Ask the question for the sake of asking the question (even especially if you don’t have any real questions). Sometimes the questions devolve into verbal diarrhea and don’t remain questions and don’t also have any meaning that can be derived from them (I don’t have a proper word to describe this state of affairs, but it is like those things which you know when you see it). But you have to open your mouth at these events, especially when you have nothing substantial/meaningful to say. This is how you get recognition. Over a decade of attending various conferences on education in India, I have come to realise that it is akin to a cartel. You go to any conference, you will see a fixed set of people who are common to these conferences. Many of these participants are the cosints (both the established and the wannabes). After spending some time in the system they become organisers of such conferences, seminars and lectures definitely get other CosInts to these conferences. These are physical citation rings, I call you to my conference you call me to yours. Year after year, I see the same patterns, so much so I can predict, like while watching a badly written and cliche movie, what is going to happen when they are around. That person has to ask a question and must use a particular buzzword. (I myself don’t ask or comment, unless I think I have something substantial to add. Perhaps they think in same manner, just that their definition of substantial is different than mine.) Also, see #5, use the terms in #1, #2 and #3. Make sure to make a personal connection with all the powerful and famous you find there, also see #4.
7. Pedigree matters: Over the years, I have seen the same type of cosints coming from particular institutions. Just like you can predict certain traits of a dog when you know its breed, similarly one can predict certain traits of individuals coming from certain institutions. Almost without exception, one can do this, but certain institutions have a greater frequency of cosints. Perhaps because the teachers who are in those places are themselves IYI+cosints. Teaching strictly from a  prescribed curriculum and rote-learning the jargon: most students just repeat what they see and the cycle continues. Sometimes I think these are the very institutions that are responsible for the sorry state of affairs in the country. They are filled to the brim with IYIs, who do not have any skin in the game and hence it doesn’t matter what they do. Also, being stamped as a product of certain institution gives you some credibility automatically, “She must be talking some sense, after all he is from DU/IIT/IIM/JNU/”
8. Quantity not quality: Most of us are not going to create work which will be recognised the world over (Claude Shannon published very infrequently, but when he did it changed the world). Yet were are in publish or perish world. CosInts know this, so they publish a lot. It doesn’t matter what is the quality is (also #4 and #5 help a lot). They truly are environmentalists. They will recycle/reuse the same material with slight changes for different papers and conferences, and surprisingly they also get it there (also #4 and #5 help a lot). So, at times, you will find a publication list which even a toilet paper roll may not be able to contain. Pages after pages of publications! Taleb’s thoughts regarding this are somewhat reassuring, so is the Sokal’s hoax, that just when someone has publications (a lot of them) it is not automatic that they are meaningful.
9. Empathisers and hypocrites: Cosints are excellent pseudo-emphatisers. They will find something to emphathise with. Maybe a class of people, a class of gender (dog only knows how many). Top of the list are marginalised, poor low socio-economic status, underprivileged, rural schools, government students, school teachers, etc. You get the picture.  They will use the buzz words in the context of these entities they emphathise with. Perhaps, once in their lifetimes, they might have visited those whom they want to give their empathy, but otherwise, it is just an abstract entity/concept.(I somehow can’t shake image of Arshad Warsi in MunnaBhai MBBS “Poor hungry people” while writing about this.) It is easier to work with abstract entities than with real ones, you don’t have to get your hands (or other body parts) dirty. The abstract teacher will do this, will behave in this way: they will write a 2000 word assignment on a terse subject. This is all good when designing things because abstract concepts don’t react in unwanted ways. But when things don’t go as planned in real world, teachers don’t react at all! The blame is on everyone else except the cosints. Perhaps they are too dumb to understand that it is they are at fault. Also, since they don’t have skin in the game, they will tell and advise whatever they have heard or think to be good, when it is implemented on others. For example, if you talk to people especially from villages, they will want to learn English as it is seen as the language which will give them upward mobility. But cosints, typically in IYI style, some researchers found that it is indeed the mother tongue which is better for students to learn, it should be implemented everywhere. The desires and hopes of those who will be learning be damned, they are too “uneducated” to understand what they need. It is the tyranny of fake experts at work here.

He thinks people should act according to their best interests and he knows their interests… When plebeians do something that makes sense to themselves, but not to him, the IYI uses the term “uneducated.” (SITG Taleb)
Now one would naturally want to know under what conditions that research was done? was there any ideological bias of the researchers? whether it is applicable in as diverse a country as India? What do we do of local “dialects”? But they don’t do any of this. Instead, they will attack anyone who raises these doubts, especially in #6. They want to work only with the government schools: poor kids, poor teachers no infrastructure. But ask them where their own children study: they do in private schools! But their medium must be their mother tongue right? No way, it is completely English medium, they even learn Hindi in English. But at least the state board? No CBSE, or still better ICSE. Thus we see the hypocrisy of the cosint, when they have the skin in the game. But do they see it themselves? Perhaps not, hence they don’t feel any conflict in what they do.

So we see that IYI /cosint are not what they seem or consider themselves. Over the last decade or so, with the rise of the right across the world is indicating to everyone that something is wrong when cosints tell us what to do. The tyranny of pseudo-experts has to go.  But why it has come to that the “intellectuals” who are supposed to be the cream of the human civilisation, the thinkers, the ideators, so why the downfall? Let us first look at the meaning of the term, so as to be not wrong about that:

The intellectual person is one who applies critical thinking and reason in either a professional or a personal capacity, and so has authority in the public sphere of their society; the term intellectual identifies three types of person, one who:

1. is erudite, and develops abstract ideas and theories;
2. a professional who produces cultural capital, as in philosophy, literary criticism, sociology, law, medicine, science; and
3. an artist who writes, composes, paints and so on.

Intellectual (emphasis mine)

Now, see in the light of the above definition, it indeed seems that it must be requiring someone to be intelligent and/or well-cultured individual. So why the change in the tones now? The reasons are that the actual intellectual class has degraded and cosints have replaced them, also too much theory and no connect with the real world has made them live in a simulacrum which is inhabited and endorsed by other cosints. And as we have seen above it is a perpetuating cycle, running especially in the universities (remember Taleb’s qualification). They theorize and jargonise (remember the buzzwords) simple concepts so much that no one who has got that special glossary will understand it). And cosints think it is how things should be. They write papers in education, supposedly for the betterment of the classroom teaching by the teachers, in such a manner that if you give it to a teacher, they will not be able to make any sense of it, leave alone finding something useful. Why? Because other cosints/IYI demand it! If you don’t write a paper in a prescribed format it is rejected, if it doesnt have enough statistics it is rejected, if it doesn’t give enough jargon in the form of theoretical review, and back scratching in the form of citations it is rejected. So what good are such papers which don’t lead to practice? And why should the teachers listen to you if you don’t have anything meaningful to tell them or something they don’t know already?
The noun to describe them:
sciolist – (noun) – One who engages in pretentious display of superficial knowledge.

# To be different

As they say in the United States: “to be different is to be indecent.” The mass crushes beneath it everything that is different, everything that is excellent, individual, qualified and select. Anybody who is not like everybody, who does not think like everybody, runs the risk of being eliminated. And it is clear, of course, that this “everybody” is not “everybody.” “Everybody” was normally the complex unity of the mass and the divergent, specialized minorities. Nowadays, “everybody” is the mass alone. Here we have the formidable fact of our times, described without any concealment of the brutality of its features.

– The Revolt of the Masses by José Ortega y Gasset.

# The Villain

Can’t but help post this little gem from an eleventh-century Sanskrit author by the name of Kshemendra. This is the first chapter of his book Desopadesa which is a satirical work on different types of base people in the society. One can’t stop from making the comparison of people around us, particularly those in power with what he describes.

## The Villain

Salutations to the villain. He is like a mortar: full of chaff as well as grain, and always fit for crushing both. Friend and foe are the same to him, as are respect and derision, and he is practised at bypassing rules. Thus is he ordained for salvation/ but he is also vile, like a dog: greedy for crumbs, fierce in quarrels and always dirty. His tongue pollutes the worthy as the dog’s does the bowl. In tardiness, malevolence and harming good works out of ill will, he is like the planet Saturn. Strangely, he is also that planet’s opposite: a thunderbolt that strikes mankind. (5-8)
Though a fool devoid of sacred learning, the villain claims to be a scholar because of his past good deeds. In extolling his own merits he is like Shesha, the thousand-headed serpent, and in running down others, like Brihaspati, the guru of the gods.
His throat is so afflicted with jealousy that his tongue cannot utter words of praise for the good, even if it is pulled out with a pair of tongs; though in slandering them, he has eyes and mouths on every side. His ears, too, are everywhere, and he hears all as he bides his time. (9-11)
The villain is like the world: illusory by nature; afflicted by passion, hatred and craziness; deluding even great minds. Whom has he not corrupted? Like a person’s pubic parts, he is, in fact, a source of shame, addiction and infatuation, and an instigator of desire. (12-13)
Ignoring his own and another’s food, the wretch always sits close to his patron, whispering slander into his ear as if it were the cosmic science. Indeed, he talks of everyone’s faults. But who talks of his? For who will ever discuss the blemishes in a dirty garment? As if in sport, the trickster even creates pictures in the sky: But he is still considered base, for among the tall he remains puny: (14-16)

With a villain, influential,
mad for money, base and cruel,
holding high office,
O people, alas, where will you go? (17)

Yet, a villainous fool is preferable to a clever villain, just as a toothless snake is to a deadly serpent, black and winged. Pollution follows the villain as it did the ogre Khara. Both are spoilers of human habitation; arrogant and hostile to the learned; devourers of mankind. Should a villain tum, by some stroke of luck, into a sincere and good person, it would be like an ape in the forest turning to prayer with its arms upraised. (18-20)
To say that a villain will praise merit is questionable, that he will love, unreasonable, and that he will give something, quite meaningless. But to say that he will kill cannot be an untruth. Influencing the master by whispering slanders in his ears night and day, I believe he spreads his control everywhere. What is the worth of anything, in the course of getting which the dust from his chamber door will adorn one’s head? It can only be a defect, never a merit. Arrogant with a bit of money, given to grand talk, the villain is a strange invention of the Creator. With eyebrows raised, he maligns, in public gatherings, the reputations of good men, which are as radiant as the expanse of Mount Kailasa.
(21-24)

# They Thought They Were Free

In this post we will look at some experiences that people in Germany had during the rise of Nazi Party. Overall the trend is that you make it almost impossible for anyone opposed to your thought as an outcast, and others just follow the herd. Many measures of the present incumbent have parallels to this. And especially the current drama of demonetization of high denomination currency notes.

This separation of government from people, this widening of the gap, took place so gradually and so insensibly, each step disguised (perhaps not even intentionally) as a temporary emergency measure or associated with true patriotic allegiance or with real social purposes. And all the crises and reforms (real reforms, too) so occupied the people that they did not see the slow motion underneath, of the whole process of government growing remoter and remoter.”

They say that it was essential that it should have been kept secret, otherwise the point of exercise would have been futile. People of the country are asked to make sacrifices for the betterment of the country. Otherwise the country was in crisis. So we had to take emergency steps. What is happening in all this introduced chaos is the issues which need to go in public imagination are removed. These are issues which the government doesn’t want people to discuss, debate. Like a magician they are directing the public attention with gimmicks and shenanigans when their slight of hand remains invisible from public scrutiny.
In all these perception managing exercise the ever changing breaking news in our main-stream-media plays an ubiquitous role. They are supposed to be a pillar in the democratic process, but instead we find that they are malleable and play hand-maiden’s role for diverting and capturing public imagination. Most of the time this is in sync with what the incumbent government wants.

“The dictatorship, and the whole process of its coming into being, was above all diverting. It provided an excuse not to think for people who did not want to think anyway. I do not speak of your ‘little men,’ your baker and so on; I speak of my colleagues and myself, learned men, mind you. Most of us did not want to think about fundamental things and never had. There was no need to. Nazism gave us some dreadful, fundamental things to think about—we were decent people—and kept us so busy with continuous changes and ‘crises’ and so fascinated, yes, fascinated, by the machinations of the ‘national enemies,’ without and within, that we had no time to think about these dreadful things that were growing, little by little, all around us. Unconsciously, I suppose, we were grateful. Who wants to think?”

Thinking is also outsourced. Experts are called in, to provide excerpts from ideas too complex and too common for common citizens to comprehend. Each idea is digested in studios, what is generated is a pre-digested version of the ideas so that you don’t need to do it. You perhaps do not have time to do it. There are more relevant things than ruminating about rationalisations regarding political policies. And if at all you do question or think about these, one of the basic logical fallacy of ad hominem is employed. Shoot the messenger, we already have the message (or massage after McLuhan). Messenger is the mess-maker. Here in public imagination the questioner becomes the questioned. The questions are irrelevant, motive, history and ideological stance of the person asking the question is more important. Questioning policies and performance metamorphose from act of trying to understand to act of treason to undermine.
The perpetrator becomes predated. Overnight they are condemned to become public and hence national enemies. Any one who does not support becomes anti-national by default. To live here you have to live by our rules, otherwise you should go away. Who gave this authority to them? This is again questioned back, you must have something to hide, hence you are not supporting this. Then it captures public imagination, those questioning are enemies within. Dissent is treason.
And we have in form of Pakistan the “Other”. The national enemy without. When there is a dullness in the public imagination, raise the ante in form of the bogey man for all our troubles. Again here the pattern is well laid out.

Uncertainty is a very important factor, and, instead of decreasing as time goes on, it grows. Outside, in the streets, in the general community, ‘everyone’ is happy. One hears no protest, and certainly sees none.

What might happen next, what event will break the news, tomorrow is unknown. Through surveys through debates it is brought to fore that “All is well.” If it is not well for you, the trouble is with you. All the problems are only for people who are enemies within. Those supporting, are the ones who are honest, happy and hardworking. Rest of you need to prove you allegiance, we already have by token sloganeering, literally and figuratively both.

And you are an alarmist. You are saying that this must lead to this, and you can’t prove it. These are the beginnings, yes; but how do you know for sure when you don’t know the end, and how do you know, or even surmise, the end? On the one hand, your enemies, the law, the regime, the Party, intimidate you. On the other, your colleagues pooh-pooh you as pessimistic or even neurotic. You are left with your close friends, who are, naturally, people who have always thought as you have.”