Dimensionality reduction without compromising integrity | Uri Itai | Aug 2024

SeniorTechInfo
3 Min Read

The Importance of Innovation in Job Interviews: Lessons Learned

Uri Itai

Recently, I found myself at the final stage of a hiring process with a renowned medical research firm. Having received positive feedback in the earlier rounds, I was optimistic and eager to showcase my expertise. However, the final interview took an unexpected and challenging turn that left me re-evaluating the entire experience.

The interviewer’s demeanor was markedly different from what I had encountered previously, almost bordering on rude. He grilled me intensely about my career choices and past projects in biological research, with a critical tone that was hard to ignore. When I shared my experience with a liquid biopsy startup, the conversation quickly became highly technical.

An unsuccessful job interview
An unsuccessful job interview

I explained how we managed an overwhelming number of features through dimensional reduction using a process called saturation. The interviewer interrupted, questioning why we hadn’t used Principal Component Analysis (PCA).

My honest response — that PCA hadn’t worked for us — was met with skepticism. “You didn’t use it properly,” he asserted.

To defend our approach, I highlighted the limitations of PCA in biological contexts. PCA essentially performs a general rotation of the data, functioning as a linear transformation that preserves both orientation and the Euclidean metric.

  • PCA’s Assumption of Euclidean Structure

PCA assumes that the data has an inherent Euclidean structure, a condition that is rarely met in biological systems. Biological data typically exhibit complex, non-Euclidean relationships that cannot be accurately represented by simple linear transformations.

  • The Issue with Linear Combinations

In biological contexts, linear combinations of variables often lack meaningful interpretation. For instance, what practical insight can be gained by adding twice the body temperature to three times the blood pressure level?

  • Non-linear Feedback Loops

Many biological systems are characterized by non-linear feedback loops, which PCA is ill-equipped to capture.

  • Sensitivity to Non-linear Scaling

PCA is not invariant to non-linear scaling or changes in units. For example, a log transformation could significantly alter PCA results, leading to different interpretations.

Reflecting on the interview, I realized the importance of finding a workplace that values critical thinking and the ability to challenge established norms when necessary. The right opportunity should not only appreciate my technical skills but also my willingness to think outside the box and apply unconventional solutions to unique problems.

In the end, not getting this job might have been a blessing in disguise. It’s a reminder that the right fit is not just about the prestige of the firm but also about mutual respect and a shared vision for innovation and progress. Moving forward, I’ll be more mindful of aligning with organizations that value a holistic and open-minded approach to problem-solving in the ever-evolving field of biological research.

Share This Article
Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *