We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

Transformers-based models, such as BERT, have been one of the most successful deep learning models for NLP. Unfortunately, one of their core limitations is the quadratic dependency (mainly in terms of memory) on the sequence length due to their full attention mechanism. To remedy this, we propose, BigBird, a sparse attention mechanism that reduces this quadratic dependency to linear. We show that BigBird is a universal approximator of sequence functions and is Turing complete, thereby preserving these properties of the quadratic, full attention model. Along the way, our theoretical analysis reveals some of the benefits of having $O(1)$ global tokens (such as CLS), that attend to the entire sequence as part of the sparse attention mechanism. The proposed sparse attention can handle sequences of length up to 8x of what was previously possible using similar hardware. As a consequence of the capability to handle longer context, BigBird drastically improves performance on various NLP tasks such as question answering and summarization. We also propose novel applications to genomics data.

pdf/big-bird-transformers-for-longer-sequences-2nkj9oqgmy.pdf

Big Bird: Transformers for Longer Sequences

An Introduction to Random Matrices: Introduction

We consider large random matrices with a general slowly decaying correlation among its entries We prove universality of the local eigenvalue statistics and optimal local laws for the resolvent away from the spectral edges, generalizing the recent result of [arXiv:160408188] to allow slow correlation decay and arbitrary expectation The main novel tool is a systematic diagrammatic control of a multivariate cumulant expansion

/pdf/random-matrices-with-slow-correlation-decay-4ge4bymgim.pdf

Random Matrices with Slow Correlation Decay

We consider the nonlinear equation $-\frac{1}{m}=z+Sm$ with a parameter $z$ in the complex upper half plane $\mathbb{H} $, where $S$ is a positivity preserving symmetric linear operator acting on bounded functions. The solution with values in $ \mathbb{H}$ is unique and its $z$-dependence is conveniently described as the Stieltjes transforms of a family of measures $v$ on $\mathbb{R}$. In [AEK17a] we qualitatively identified the possible singular behaviors of $v$: under suitable conditions on $S$ we showed that in the density of $v$ only algebraic singularities of degree two or three may occur. In this paper we give a comprehensive analysis of these singularities with uniform quantitative controls. We also find a universal shape describing the transition regime between the square root and cubic root singularities. Finally, motivated by random matrix applications in the companion paper [AEK16b], we present a complete stability analysis of the equation for any $z\in \mathbb{H}$, including the vicinity of the singularities.

/pdf/quadratic-vector-equations-on-complex-upper-half-plane-4s0uc7ghqj.pdf

Quadratic vector equations on complex upper half-plane

We study the unique solution $m$ of the Dyson equation \[ -m(z)^{-1} = z - a + S[m(z)] \] on a von Neumann algebra $\mathcal{A}$ with the constraint $\mathrm{Im}\,m\geq 0$. Here, $z$ lies in the complex upper half-plane, $a$ is a self-adjoint element of $\mathcal{A}$ and $S$ is a positivity-preserving linear operator on $\mathcal{A}$. We show that $m$ is the Stieltjes transform of a compactly supported $\mathcal{A}$-valued measure on $\mathbb{R}$. Under suitable assumptions, we establish that this measure has a uniformly $1/3$-Holder continuous density with respect to the Lebesgue measure, which is supported on finitely many intervals, called bands. In fact, the density is analytic inside the bands with a square-root growth at the edges and internal cubic root cusps whenever the gap between two bands vanishes. The shape of these singularities is universal and no other singularity may occur. We give a precise asymptotic description of $m$ near the singular points. These asymptotics generalize the analysis at the regular edges given in the companion paper on the Tracy-Widom universality for the edge eigenvalue statistics for correlated random matrices [arXiv:1804.07744] and they play a key role in the proof of the Pearcey universality at the cusp for Wigner-type matrices [arXiv:1809.03971,arXiv:1811.04055]. We also extend the finite dimensional band mass formula from [arXiv:1804.07744] to the von Neumann algebra setting by showing that the spectral mass of the bands is topologically rigid under deformations and we conclude that these masses are quantized in some important cases.

The Dyson equation with linear self-energy: spectral bands, edges and cusps

We complete the analysis of the extremal eigenvalues of the adjacency matrix A of the Erdős–Renyi graph G(N,d/N) in the critical regime d≍logN of the transition uncovered in (Ann. Inst. Henri Poincare Probab. Stat. 56 (2020) 2141–2161; Ann. Probab. 47 (2019) 1653–1676), where the regimes d≫logN and d≪logN were studied. We establish a one-to-one correspondence between vertices of degree at least 2d and nontrivial (excluding the trivial top eigenvalue) eigenvalues of A/ d outside of the asymptotic bulk [−2,2]. This correspondence implies that the transition characterized by the appearance of the eigenvalues outside of the asymptotic bulk takes place at the critical value d=d∗=1log4−1logN. For d d∗, we show that no such eigenvalues exist. All of our estimates are quantitative with polynomial error probabilities.
Our proof is based on a tridiagonal representation of the adjacency matrix and on a detailed analysis of the geometry of the neighbourhood of the large degree vertices. An important ingredient in our estimates is a matrix inequality obtained via the associated nonbacktracking matrix and an Ihara–Bass formula (Ann. Inst. Henri Poincare Probab. Stat. 56 (2020) 2141–2161). Our argument also applies to sparse Wigner matrices, defined as the Hadamard product of A and a Wigner matrix, in which case the role of the degrees is replaced by the squares of the l2-norms of the rows.

Extremal eigenvalues of critical Erdős–Rényi graphs

We prove edge universality for a general class of correlated real symmetric or complex Hermitian Wigner matrices with arbitrary expectation. Our theorem also applies to internal edges of the self-consistent density of states. In particular, we establish a strong form of band rigidity which excludes mismatches between location and label of eigenvalues close to internal edges in these general models.

Correlated random matrices: Band rigidity and edge universality

We consider random $n\times n$ matrices $X$ with independent and centered entries and a general variance profile. We show that the spectral radius of $X$ converges with very high probability to the square root of the spectral radius of the variance matrix of $X$ when $n$ tends to infinity. We also establish the optimal rate of convergence, that is a new result even for general i.i.d. matrices beyond the explicitly solvable Gaussian cases. The main ingredient is the proof of the local inhomogeneous circular law [arXiv:1612.07776] at the spectral edge.

Spectral radius of random matrices with independent entries

We analyse the eigenvectors of the adjacency matrix of a critical Erdős–Renyi graph $${\mathbb {G}}(N,d/N)$$
 , where d is of order $$\log N$$
 . We show that its spectrum splits into two phases: a delocalized phase in the middle of the spectrum, where the eigenvectors are completely delocalized, and a semilocalized phase near the edges of the spectrum, where the eigenvectors are essentially localized on a small number of vertices. In the semilocalized phase the mass of an eigenvector is concentrated in a small number of disjoint balls centred around resonant vertices, in each of which it is a radial exponentially decaying function. The transition between the phases is sharp and is manifested in a discontinuity in the localization exponent $$\gamma (\varvec{\mathrm {w}})$$
 of an eigenvector $$\varvec{\mathrm {w}}$$
 , defined through $$\Vert \varvec{\mathrm {w}} \Vert _\infty / \Vert \varvec{\mathrm {w}} \Vert _2 = N^{-\gamma (\varvec{\mathrm {w}})}$$
 . Our results remain valid throughout the optimal regime $$\sqrt{\log N} \ll d \leqslant O(\log N)$$
 .

/pdf/delocalization-transition-for-critical-erdos-renyi-graphs-kyaieltjmr.pdf

Delocalization Transition for Critical Erdős–Rényi Graphs

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Johannes Alt

Author Tools

Chat about Author